spyros
/
dnet-hadoop
forked from D-Net/dnet-hadoop
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
697 Commits 13 Branches 0 Tags 63 MiB
Commit Graph

588 Commits

Author SHA1 Message Date
Miriam Baglioni 31ea05297d moved the DbClient to common and added needed dependency to pom 2020-05-04 12:22:28 +02:00
Miriam Baglioni dbf3ba051a minor 2020-04-30 20:22:07 +02:00
Miriam Baglioni 43053a286d workflow pom with added blacklist module 2020-04-30 18:30:21 +02:00
Miriam Baglioni 0631fe548a pom.xml 2020-04-30 18:29:46 +02:00
Miriam Baglioni 38ecfd5785 the wf with all the three steps for blacklisting relations 2020-04-30 18:28:46 +02:00
Miriam Baglioni 95433e1087 parameters for the preparation phase and blacklist phase 2020-04-30 18:28:13 +02:00
Miriam Baglioni 1070790c19 minor 2020-04-30 18:26:58 +02:00
Miriam Baglioni b9d56b3ced applies the actual removal of the relations 2020-04-30 18:26:25 +02:00
Miriam Baglioni d6d6ebeae5 preparation step: creates the subset of the merges relations 2020-04-30 18:25:33 +02:00
Miriam Baglioni 276b95b7b3 add create file instruction 2020-04-30 15:05:17 +02:00
Miriam Baglioni 354f0162be changes in the blacklist and workflow definition 2020-04-30 10:26:50 +02:00
Miriam Baglioni 6a47e6191d read from blacklist and write the result as relations on hdfs 2020-04-29 18:16:01 +02:00
Miriam Baglioni 869f576273 added hash map for relationship entityType id prefix, and relation inverse 2020-04-29 18:14:52 +02:00
Miriam Baglioni b85ad7012a reads the blacklist from the blacklist db and writes it as a set of relations on hdfs 2020-04-29 17:29:49 +02:00
Miriam Baglioni f7695e833c resolved conflicts 2020-04-29 11:41:31 +02:00
Claudio Atzori 964972d29a added data provision workflow definition WIP 2020-04-29 09:25:50 +02:00
miconis 62e467eb0c assertion numbers updated to fit the new implementation of the pace-core 2020-04-28 11:46:23 +02:00
Claudio Atzori 6f5b899038 reformatted code according to the updated style descriptor 2020-04-28 11:23:29 +02:00
Claudio Atzori ac25f2d8d1 integrated changes from master 2020-04-28 08:55:28 +02:00
Miriam Baglioni 2980e50edf merge upstream 2020-04-27 15:06:48 +02:00
Claudio Atzori a0bdbacdae switched automatic code formatting plugin to net.revelc.code.formatter:formatter-maven-plugin 2020-04-27 14:52:31 +02:00
Claudio Atzori 7a3f8085f7 switched automatic code formatting plugin to net.revelc.code.formatter:formatter-maven-plugin 2020-04-27 14:45:40 +02:00
Michele Artini 1260d03eba skip empty projects 2020-04-27 13:51:13 +02:00
Miriam Baglioni df34a4ebcc changed the configuration to add ignorecase option to each verb related to covid-19 community 2020-04-27 12:32:56 +02:00
Miriam Baglioni 7a59324ccf changed the test to check for the new ignorecase option 2020-04-27 12:31:46 +02:00
Miriam Baglioni 986c97348d added the ignorecase option to each selection verb 2020-04-27 12:31:05 +02:00
Miriam Baglioni a303fc9f73 resources for testing propagation of result to comminuty from organization and from semrel 2020-04-27 11:14:16 +02:00
Miriam Baglioni c093d764a3 - 2020-04-27 11:12:38 +02:00
Miriam Baglioni c925e2be16 test for propagation of result to community from organization and result to community from semrel 2020-04-27 10:59:53 +02:00
Miriam Baglioni ec7f166690 changed the bl because of changed of the examples for the re implementation of the propagation step 2020-04-27 10:58:41 +02:00
Miriam Baglioni 6135096ef1 refactoring 2020-04-27 10:57:50 +02:00
Miriam Baglioni d30e710165 fixed duplicates action name in the workflow 2020-04-27 10:52:30 +02:00
Miriam Baglioni f9ee343fc0 new parametrized workflow with preparation steps and new parameter input files 2020-04-27 10:48:31 +02:00
Miriam Baglioni e2093644dc changed in the workflow the directory where to store the preparedInfo and the graph genearated at this step 2020-04-27 10:46:44 +02:00
Miriam Baglioni 8a58bf2744 removed the writeUpdate option 2020-04-27 10:45:06 +02:00
Miriam Baglioni 5dccbe13db merge with upstream 2020-04-27 10:43:59 +02:00
Miriam Baglioni 7b6505ec69 new resuorces for testing propagation of project to result after the re-implementation 2020-04-27 10:42:16 +02:00
Miriam Baglioni 1b0e0bd1b5 refactoring 2020-04-27 10:40:26 +02:00
Miriam Baglioni e5a177f0a7 refactoring 2020-04-27 10:36:21 +02:00
Miriam Baglioni e000754c92 refactoring 2020-04-27 10:34:03 +02:00
Miriam Baglioni 95a54d5460 removed the writeUpdate option. The update is available in the preparedInfo path 2020-04-27 10:30:32 +02:00
Miriam Baglioni 8802e4126b re-implemented inverting the couple: from (projectId, relatedResultList) to (resultId, relatedProjectList) 2020-04-27 10:26:55 +02:00
Claudio Atzori 268462623a refined definition of equals and hash methods for Oaf model classes, now based on entity identifier, while relations consider sourceid, targetid and relationship semantic; Factored out function to group Oaf objects in grouping operations; Raw graph creation procedure merges entities and relationships providing the same identity 2020-04-24 14:42:01 +02:00
Claudio Atzori a3e480d1c9 implmented DispatchEntitiesApplication using spark2 datasets 2020-04-24 14:36:53 +02:00
Claudio Atzori 48157e0fc4 GraphHiveImporterJob moved in dedicate package 2020-04-24 14:32:28 +02:00
Miriam Baglioni adcbf0e29a refactoring 2020-04-24 10:47:43 +02:00
Claudio Atzori 278fc9d276 code formatting 2020-04-23 18:51:38 +02:00
miconis 5414236644 Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop 2020-04-23 18:17:23 +02:00
miconis 8d258c85ff spark dedup test fixed, sample for dataset and orp added, test implemented 2020-04-23 18:16:20 +02:00
Michele Artini 072eae3803 fixed a problem with missing contexts 2020-04-23 16:42:49 +02:00