sabeel
/
dnet-hadoop
forked from D-Net/dnet-hadoop
1
0
Fork 0
Code Pull Requests Activity
814 Commits 55 Branches 18 Tags 649 MiB
Commit Graph

814 Commits

Author SHA1 Message Date
Miriam Baglioni d30e710165 fixed duplicates action name in the workflow 2020-04-27 10:52:30 +02:00
Miriam Baglioni f9ee343fc0 new parametrized workflow with preparation steps and new parameter input files 2020-04-27 10:48:31 +02:00
Miriam Baglioni e2093644dc changed in the workflow the directory where to store the preparedInfo and the graph genearated at this step 2020-04-27 10:46:44 +02:00
Miriam Baglioni 8a58bf2744 removed the writeUpdate option 2020-04-27 10:45:06 +02:00
Miriam Baglioni 5dccbe13db merge with upstream 2020-04-27 10:43:59 +02:00
Miriam Baglioni 7b6505ec69 new resuorces for testing propagation of project to result after the re-implementation 2020-04-27 10:42:16 +02:00
Miriam Baglioni 1b0e0bd1b5 refactoring 2020-04-27 10:40:26 +02:00
Miriam Baglioni e5a177f0a7 refactoring 2020-04-27 10:36:21 +02:00
Miriam Baglioni e000754c92 refactoring 2020-04-27 10:34:03 +02:00
Miriam Baglioni 95a54d5460 removed the writeUpdate option. The update is available in the preparedInfo path 2020-04-27 10:30:32 +02:00
Miriam Baglioni 8802e4126b re-implemented inverting the couple: from (projectId, relatedResultList) to (resultId, relatedProjectList) 2020-04-27 10:26:55 +02:00
Claudio Atzori fad94c2155 updated dependency dnet-pace-core to version 4.0.1 to include 7bc00a3f5f 2020-04-24 16:47:10 +02:00
Claudio Atzori 268462623a refined definition of equals and hash methods for Oaf model classes, now based on entity identifier, while relations consider sourceid, targetid and relationship semantic; Factored out function to group Oaf objects in grouping operations; Raw graph creation procedure merges entities and relationships providing the same identity 2020-04-24 14:42:01 +02:00
Claudio Atzori a3e480d1c9 implmented DispatchEntitiesApplication using spark2 datasets 2020-04-24 14:36:53 +02:00
Claudio Atzori 48157e0fc4 GraphHiveImporterJob moved in dedicate package 2020-04-24 14:32:28 +02:00
Miriam Baglioni adcbf0e29a refactoring 2020-04-24 10:47:43 +02:00
Claudio Atzori 5100527400 added default value for resulttype field 2020-04-23 19:14:37 +02:00
Claudio Atzori 278fc9d276 code formatting 2020-04-23 18:51:38 +02:00
miconis 5414236644 Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop 2020-04-23 18:17:23 +02:00
miconis 8d258c85ff spark dedup test fixed, sample for dataset and orp added, test implemented 2020-04-23 18:16:20 +02:00
Michele Artini 072eae3803 fixed a problem with missing contexts 2020-04-23 16:42:49 +02:00
Michele Artini b164d96874 Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop 2020-04-23 16:19:16 +02:00
Michele Artini d920ce501e fixed a problem with missing instances 2020-04-23 16:18:40 +02:00
Miriam Baglioni 0e447add66 removed unuseful classes 2020-04-23 12:59:43 +02:00
Miriam Baglioni edb00db86a refactoring 2020-04-23 12:57:35 +02:00
Miriam Baglioni 44fab140de - 2020-04-23 12:42:07 +02:00
Miriam Baglioni 769aa8178a refactoring 2020-04-23 12:40:44 +02:00
Miriam Baglioni d8dc31d4af refactoring 2020-04-23 12:35:49 +02:00
Miriam Baglioni 8c5dac5cc3 removed unuseful classes 2020-04-23 12:30:58 +02:00
Miriam Baglioni 15656684b9 added proeprties for the preparation step and actual propagation. Added the new parametrized workflow 2020-04-23 12:13:34 +02:00
Miriam Baglioni 6f35f5ca42 added the steps of reset output dir and copy information not changed by the propagation step 2020-04-23 12:12:07 +02:00
Miriam Baglioni 19cd5b85c0 changed the classname to execute 2020-04-23 12:07:41 +02:00
Miriam Baglioni fa2ff5c6f5 refactoring 2020-04-23 11:58:26 +02:00
Miriam Baglioni 540f70298b added missing property 2020-04-23 11:51:48 +02:00
Miriam Baglioni e431fe4f5b added the implements Serializable to each class 2020-04-23 11:48:47 +02:00
Miriam Baglioni 24fa81d7e8 implementation parametrized for result type 2020-04-23 11:44:19 +02:00
Miriam Baglioni ac7ec349cf removed the shaded lib module 2020-04-23 11:43:03 +02:00
Miriam Baglioni ab2a24cc2b changed the dependency to use reflections to find annotated classes 2020-04-23 11:08:47 +02:00
Miriam Baglioni 04fc223346 add method addPid 2020-04-23 11:07:44 +02:00
Miriam Baglioni 5153d88bd3 defiition of workflow and properties for bulktagging 2020-04-23 11:04:53 +02:00
Miriam Baglioni 3b2e4ab670 test for bulktag 2020-04-23 10:00:10 +02:00
Claudio Atzori 8851050814 replaced hive_db_name with hiveDbName 2020-04-23 08:36:40 +02:00
Claudio Atzori 91f81107b1 applying code formatting 2020-04-23 07:52:32 +02:00
Claudio Atzori 1e7583c5a6 filtered invisible records in data provision workflow 2020-04-23 07:51:34 +02:00
Claudio Atzori 9ddafd46ca fixed dedup record id prefix, set the correct dataInfo in the DedupRecordFactory 2020-04-23 07:50:18 +02:00
Claudio Atzori ade4cb97af fixed parameters passed to the postprocessing action in the workflow mapping the graph as hive DB 2020-04-22 18:24:06 +02:00
Claudio Atzori ba4339f142 excluded org.apache.hadoop:hadoop-common from the dnet-actionmanager-common dependency to avoid multiple transitive jaxb-impl versions to conflict when instantiating the ISLookup client stub 2020-04-22 14:23:09 +02:00
Claudio Atzori e81960335c Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop 2020-04-22 10:46:37 +02:00
Michele Artini 9e4d58f505 ResultType 2020-04-22 10:07:26 +02:00
Claudio Atzori c891661822 small adjustments in the graph2hive workflow 2020-04-21 18:52:23 +02:00