sabeel
/
dnet-hadoop
forked from D-Net/dnet-hadoop
1
0
Fork 0
Code Pull Requests Activity
835 Commits 55 Branches 18 Tags 649 MiB
Commit Graph

835 Commits

Author SHA1 Message Date
Miriam Baglioni 15656684b9 added proeprties for the preparation step and actual propagation. Added the new parametrized workflow 2020-04-23 12:13:34 +02:00
Miriam Baglioni 6f35f5ca42 added the steps of reset output dir and copy information not changed by the propagation step 2020-04-23 12:12:07 +02:00
Miriam Baglioni 19cd5b85c0 changed the classname to execute 2020-04-23 12:07:41 +02:00
Miriam Baglioni fa2ff5c6f5 refactoring 2020-04-23 11:58:26 +02:00
Miriam Baglioni 540f70298b added missing property 2020-04-23 11:51:48 +02:00
Miriam Baglioni e431fe4f5b added the implements Serializable to each class 2020-04-23 11:48:47 +02:00
Miriam Baglioni 24fa81d7e8 implementation parametrized for result type 2020-04-23 11:44:19 +02:00
Miriam Baglioni ac7ec349cf removed the shaded lib module 2020-04-23 11:43:03 +02:00
Miriam Baglioni ab2a24cc2b changed the dependency to use reflections to find annotated classes 2020-04-23 11:08:47 +02:00
Miriam Baglioni 04fc223346 add method addPid 2020-04-23 11:07:44 +02:00
Miriam Baglioni 5153d88bd3 defiition of workflow and properties for bulktagging 2020-04-23 11:04:53 +02:00
Miriam Baglioni 3b2e4ab670 test for bulktag 2020-04-23 10:00:10 +02:00
Claudio Atzori 8851050814 replaced hive_db_name with hiveDbName 2020-04-23 08:36:40 +02:00
Claudio Atzori 91f81107b1 applying code formatting 2020-04-23 07:52:32 +02:00
Claudio Atzori 1e7583c5a6 filtered invisible records in data provision workflow 2020-04-23 07:51:34 +02:00
Claudio Atzori 9ddafd46ca fixed dedup record id prefix, set the correct dataInfo in the DedupRecordFactory 2020-04-23 07:50:18 +02:00
Claudio Atzori ade4cb97af fixed parameters passed to the postprocessing action in the workflow mapping the graph as hive DB 2020-04-22 18:24:06 +02:00
Claudio Atzori ba4339f142 excluded org.apache.hadoop:hadoop-common from the dnet-actionmanager-common dependency to avoid multiple transitive jaxb-impl versions to conflict when instantiating the ISLookup client stub 2020-04-22 14:23:09 +02:00
Claudio Atzori e81960335c Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop 2020-04-22 10:46:37 +02:00
Michele Artini 9e4d58f505 ResultType 2020-04-22 10:07:26 +02:00
Claudio Atzori c891661822 small adjustments in the graph2hive workflow 2020-04-21 18:52:23 +02:00
Miriam Baglioni 259525cb93 Merge remote-tracking branch 'upstream/master' 2020-04-21 18:33:46 +02:00
Miriam Baglioni 30e53261d0 minor 2020-04-21 18:00:53 +02:00
Claudio Atzori 0b55795d4d small adjustments in the provisioning workflow 2020-04-21 16:15:04 +02:00
Claudio Atzori 88fbb3a353 added sparkSqlWarehouseDir to the default extra spark options passed to each workflow 2020-04-21 16:13:43 +02:00
Claudio Atzori cd320efa96 added extra spark options to graph to hive workflow 2020-04-21 16:12:20 +02:00
Miriam Baglioni 676ba49324 changed pom of dhp-build 2020-04-21 16:09:23 +02:00
Miriam Baglioni 90c768dde6 added shaded libs module 2020-04-21 16:03:51 +02:00
Claudio Atzori 91e72a6944 Dataset based implementation for SparkCreateDedupRecord phase, fixed datasource entity dump supplementing dedup unit tests 2020-04-21 12:06:08 +02:00
miconis 5c9ef08a8e spark dedup test fixed 2020-04-21 10:19:04 +02:00
Claudio Atzori d772d967aa restored changes from master branch 2020-04-20 18:53:06 +02:00
Claudio Atzori eb8a020859 fixed behaviour of DedupRecordFactory 2020-04-20 18:44:06 +02:00
Claudio Atzori ede1af3d85 Merge branch 'master' into deduptesting 2020-04-20 16:52:14 +02:00
miconis 1102e32462 SparkDedupTest updated and organization dump fixed 2020-04-20 16:49:01 +02:00
Claudio Atzori 667d23c58b finalising Actionset migration workflow 2020-04-20 16:45:21 +02:00
miconis 4da13e4570 Revert "Merge branch 'master' into deduptesting" 
This reverts commit 772f75d167, reversing
changes made to 5f45f2c77f.
 2020-04-20 16:04:49 +02:00
Claudio Atzori 9147af7fed actionsets migration workflow moved in dhp-workflows/dhp-actionmanager 2020-04-20 15:24:33 +02:00
miconis 772f75d167 Merge branch 'master' into deduptesting 2020-04-20 14:50:12 +02:00
Claudio Atzori d714bfb4d4 collectedfrom field moved in common parent class Oaf.java 2020-04-20 12:25:19 +02:00
Michele Artini 8ff7facfa3 fixed collectedFrom ID 2020-04-20 11:09:27 +02:00
Michele Artini 25307965d2 add a default datainfo if missing 2020-04-20 09:43:27 +02:00
Michele Artini d2058fdc47 tests 2020-04-20 09:31:14 +02:00
Michele Artini 478a958f09 tests 2020-04-20 09:15:27 +02:00
Miriam Baglioni e1848b7603 minor 2020-04-18 14:16:42 +02:00
Miriam Baglioni 0ff9b1ef05 added needed parameter 2020-04-18 14:16:29 +02:00
Miriam Baglioni e2dfe8b656 removed not used action 2020-04-18 14:16:07 +02:00
Miriam Baglioni 437ebbad76 refactorign 2020-04-18 14:15:09 +02:00
Miriam Baglioni 9a8876ac86 added needed parameter 2020-04-18 14:14:08 +02:00
Miriam Baglioni 9854852878 refactoring 2020-04-18 14:13:16 +02:00
Miriam Baglioni 454b8a6a29 Merge remote-tracking branch 'upstream/master' 2020-04-18 14:09:44 +02:00