przemyslaw.jacewicz
/
dnet-hadoop
forked from D-Net/dnet-hadoop
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
753 Commits 16 Branches 0 Tags 76 MiB
Commit Graph

639 Commits

Author SHA1 Message Date
Claudio Atzori b2192fdcdc simplified reset_outputpath nodes across the workflows, applied common xml formatting 2020-05-08 12:33:31 +02:00
Miriam Baglioni 9b4c0d4b3a - 2020-05-08 11:51:45 +02:00
Miriam Baglioni 53952707b6 modified test because of new step of data preparation. It now expects to find ResultCountrySet serialization nstead of DatasourceCountry 2020-05-08 11:49:19 +02:00
Miriam Baglioni d6b9de9f46 Merge branch 'master' of https://code-repo.d4science.org/miriam.baglioni/dnet-hadoop 2020-05-07 18:22:59 +02:00
Claudio Atzori 166aafd936 heavy cleanup 2020-05-07 18:22:26 +02:00
Miriam Baglioni fb405275f7 merged with master 2020-05-07 11:48:21 +02:00
Miriam Baglioni e124278934 - 2020-05-07 11:47:11 +02:00
Claudio Atzori 5111671e62 celanup 2020-05-07 11:47:00 +02:00
Miriam Baglioni 9f8855991c changed Encorders.bean to Encoders.kryo 2020-05-07 11:44:35 +02:00
Miriam Baglioni 207b899d6d merged with upstream 2020-05-07 11:43:53 +02:00
Claudio Atzori 5b3f8a0e90 using Encoders.bean instead of kryo 2020-05-07 11:41:41 +02:00
Miriam Baglioni 182225becb Merge branch 'master' of https://code-repo.d4science.org/miriam.baglioni/dnet-hadoop 2020-05-07 11:38:17 +02:00
Miriam Baglioni 5efae3acb9 new workflow for job3 2020-05-07 11:38:10 +02:00
Claudio Atzori 73243793b2 Dataset based implementation for SparkCountryPropagationJob3 2020-05-07 11:15:24 +02:00
Claudio Atzori 128c3bf1c8 restored Author bean with simple getter/setter, author pid addition moved into dedicated implementation SparkOrcidToResultFromSemRelJob3 2020-05-07 11:14:56 +02:00
Miriam Baglioni b2fec32c87 new workflow for job3 2020-05-07 10:01:57 +02:00
Miriam Baglioni 29bc8c44b1 changes in the construction of new country set 2020-05-07 10:01:34 +02:00
Miriam Baglioni 55e825acd4 chenged the test according to changes in SparkCOuntryPropagationJob2 2020-05-07 10:01:00 +02:00
Miriam Baglioni 16193cf0ba new workflow and parameter for country propagation 2020-05-07 09:59:58 +02:00
Miriam Baglioni 5a476c7a13 chenged the xquery for the cfhb table 2020-05-07 09:58:17 +02:00
Miriam Baglioni 42ad51577a new implementation with one more serialization step 2020-05-07 09:57:49 +02:00
Claudio Atzori 17860d3ab6 general changes in the RAW graph mapping: missing collectedfrom/hostedby causes records to be skipped; factored out most of the constants in ModelConstants class (dhp-schemas) 2020-05-06 13:20:02 +02:00
Claudio Atzori fdfecc9578 Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop 2020-05-06 11:28:01 +02:00
Claudio Atzori c79e2f5977 drop workingPath before starting the dedup workflow 2020-05-06 11:27:44 +02:00
Michele Artini 8f30a09d84 Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop 2020-05-05 17:12:22 +02:00
Michele Artini ccc609f909 new module for the production of broker events 2020-05-05 17:09:00 +02:00
Miriam Baglioni dd2e698a72 added a sequentialization step on the spark job. Addedd new parameter 2020-05-05 17:03:43 +02:00
Claudio Atzori 0825321d0b improved unit tests in dhp-aggregation 2020-05-05 12:39:04 +02:00
Miriam Baglioni 252b219dd5 chanced the name of some properties 2020-05-05 10:03:32 +02:00
Claudio Atzori 4a8487165c using long param names in wf definition 2020-05-04 19:19:29 +02:00
Claudio Atzori a2fc37df5f adjusted parameters 2020-05-04 19:18:59 +02:00
Claudio Atzori f1b7e14036 code formatting 2020-05-04 19:18:34 +02:00
Miriam Baglioni 78578c3ccf fixed wrong trnasition name in workflow 2020-05-04 15:46:24 +02:00
Miriam Baglioni cc7d9b6b19 merge upstream 2020-05-04 13:59:09 +02:00
Miriam Baglioni 3957c815b9 changed the name of some parameters 2020-05-04 13:58:52 +02:00
miconis 085cf173d7 Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop 2020-05-04 12:08:20 +02:00
miconis 3df703f67d mergerels added to propagate relations 2020-05-04 12:08:12 +02:00
Claudio Atzori bac37b3973 fixed children expansion in XML records 2020-05-04 11:51:17 +02:00
Claudio Atzori 077ccd8743 stats wf properties cleanup 2020-05-04 11:41:46 +02:00
Miriam Baglioni b7dd400e51 added check if author.pid exists or is null 2020-05-01 15:09:02 +02:00
Miriam Baglioni 13f30664ea minor 2020-04-30 15:23:49 +02:00
Miriam Baglioni 65a5d67b8b minor modifications 2020-04-30 14:45:27 +02:00
Miriam Baglioni 418595fec2 removed the saveGraph parameter 2020-04-30 14:45:00 +02:00
Miriam Baglioni ce8b1d0bc3 new workflow definition to be inserted in the provision pipeline 2020-04-30 14:38:54 +02:00
Miriam Baglioni 4b0bd91012 - 2020-04-30 12:45:28 +02:00
Miriam Baglioni 2349bfd8b8 changed the job test to remove the writeUpdate option 2020-04-30 11:43:33 +02:00
Miriam Baglioni 951517f9ec new input parameters and workflow definition to be used in the provision pipeline 2020-04-30 11:32:50 +02:00
Miriam Baglioni 026f297e49 removed the writeUpdate oprion 2020-04-30 11:31:59 +02:00
Miriam Baglioni c89fe762b1 modified relation datasource organization 2020-04-30 11:17:03 +02:00
Miriam Baglioni 3abb76ff7a merge with upstream 2020-04-30 11:15:54 +02:00