 Miriam Baglioni
|
964b22d418
|
modified the writing of the new relations. before: read old rels, add the new ones to them, write all the relations in new location. Now: first step of the wf copies the old relation i new location. If new relations are found, they are saved in the new location in append mode.
|
2020-04-15 12:32:01 +02:00 |
|
Miriam Baglioni
|
43f0590d4b
|
change in the testing because the business logic is changed.
|
2020-04-15 12:29:50 +02:00 |
|
Miriam Baglioni
|
473d17767c
|
new business logic for the actual propagation. It exploits previously computed information
|
2020-04-15 12:25:44 +02:00 |
|
Miriam Baglioni
|
6a377a7582
|
class to compute some information needed for the actual propagation
|
2020-04-15 12:25:11 +02:00 |
|
Miriam Baglioni
|
5a3487280d
|
classes to serialize/deserialize the prepared data
|
2020-04-15 12:24:36 +02:00 |
|
Miriam Baglioni
|
62b09be43c
|
added correct descritption for parameter isSparkSessionManaged
|
2020-04-15 12:23:06 +02:00 |
|
Miriam Baglioni
|
1859ce8902
|
minor refactoring
|
2020-04-15 12:21:31 +02:00 |
|
Miriam Baglioni
|
27f1d3ee8f
|
minor refactoring
|
2020-04-15 12:21:05 +02:00 |
 Alessia Bardi
|
550a9f82ed
|
Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop
|
2020-04-14 17:53:01 +02:00 |
|
Alessia Bardi
|
a68fae9bcb
|
now supporting openaire 4.0 compliance
|
2020-04-14 17:52:48 +02:00 |
 Sandro La Bruzzo
|
c36239e693
|
fixed incremental indexing
|
2020-04-14 17:47:36 +02:00 |
|
Miriam Baglioni
|
3f4b579e7f
|
new workflow. It is composed of four steps. The first removes the directory where to store the results. The second copies the relation to the new locatio, the third id the preparation phase and then the actual propagation
|
2020-04-14 16:49:24 +02:00 |
|
Miriam Baglioni
|
ca2b40952e
|
minor changes
|
2020-04-14 16:48:02 +02:00 |
|
Miriam Baglioni
|
61d39e659e
|
parameters for the project2result propagation phase
|
2020-04-14 16:47:39 +02:00 |
|
Miriam Baglioni
|
92f19fa0a0
|
parameters for the project2result preparation phase
|
2020-04-14 16:46:57 +02:00 |
|
Miriam Baglioni
|
cadab9b81d
|
new implementation for result to project propagation. Use the prepared info in propagation
|
2020-04-14 16:46:07 +02:00 |
|
Miriam Baglioni
|
ceb1f299bf
|
minor changes
|
2020-04-14 16:45:12 +02:00 |
 Claudio Atzori
|
82e8341f50
|
reorganizing parameter names in the provision workflow
|
2020-04-14 15:54:41 +02:00 |
|
Miriam Baglioni
|
e0038bde5b
|
Support class to serialize/deserialize the association project, set of linked results
|
2020-04-14 15:32:12 +02:00 |
|
Miriam Baglioni
|
c0bebb7c35
|
code to compute the prepared information used in the actual propagation step. This step will produce who files: one with potential updates (association between projects and a list of results), the other already linked entities (association between projects and the list of results already linked to them)
|
2020-04-14 15:31:26 +02:00 |
|
Miriam Baglioni
|
f47ee5b78e
|
directory where to store the prepared infor before actual propagation will take place
|
2020-04-14 15:29:21 +02:00 |
|
Miriam Baglioni
|
36cc9516d8
|
the starting relation set for testing
|
2020-04-14 15:28:34 +02:00 |
|
Miriam Baglioni
|
4b01dc60e6
|
test unit for result to project propagation
|
2020-04-14 15:28:00 +02:00 |
|
Miriam Baglioni
|
8f12292daa
|
changed the way to save the results on filesystem
|
2020-04-11 16:47:34 +02:00 |
|
Miriam Baglioni
|
87f802821e
|
new workflow for country propagation: it is composed of the preparation step and in the propagation. The propagation part runs in parallel on the result types
|
2020-04-11 16:40:22 +02:00 |
|
Miriam Baglioni
|
a562080b0b
|
parameters to be used in the prepared Job and in the actual country propagation job
|
2020-04-11 16:39:17 +02:00 |
|
Miriam Baglioni
|
1251ad4455
|
removed unuseful class
|
2020-04-11 16:38:13 +02:00 |
|
Miriam Baglioni
|
aef9b3aa90
|
new parametric implementation of country propagation. Exploits information compute before and broadcasts it to each executor
|
2020-04-11 16:36:59 +02:00 |
|
Miriam Baglioni
|
a2d833d5dd
|
step of data preparation before actual country propagation will take palce
|
2020-04-11 16:36:03 +02:00 |
|
Miriam Baglioni
|
6897c920a2
|
classes in support of new implementation of country propagation
|
2020-04-11 16:35:26 +02:00 |
|
Miriam Baglioni
|
85766a02d8
|
added dependency to use hive on local machine
|
2020-04-11 16:34:22 +02:00 |
|
Miriam Baglioni
|
79b8ea4fed
|
prepared information to be used in actual country propagation. Subset of info
|
2020-04-11 16:29:41 +02:00 |
|
Miriam Baglioni
|
1822476613
|
Test for country propagation
|
2020-04-11 16:28:09 +02:00 |
|
Miriam Baglioni
|
7783b09c5b
|
new implementation for result to project propagation. Prepare some info to be used in propagation
|
2020-04-11 16:26:23 +02:00 |
|
Claudio Atzori
|
6b5f9ca9cb
|
raw graph creation workflow moved under dhp-graph-mapper, claims integration is included
|
2020-04-10 17:53:07 +02:00 |
|
Miriam Baglioni
|
90469789b9
|
two new classes fro new implementation of project to result propagation
|
2020-04-09 13:29:01 +02:00 |
|
Miriam Baglioni
|
627ad58a8b
|
new wf definition
|
2020-04-09 11:33:19 +02:00 |
|
Miriam Baglioni
|
9c63c4840d
|
new workflow and parameters for country propagation
|
2020-04-08 19:13:42 +02:00 |
|
Miriam Baglioni
|
a2d309545b
|
new parametrized implementation for country propagation
|
2020-04-08 19:12:59 +02:00 |
|
Miriam Baglioni
|
6dfdba9ef7
|
new parametrized implementation for country propagation
|
2020-04-08 18:14:37 +02:00 |
|
Miriam Baglioni
|
03f7cb6402
|
new parametrized implementation for country propagation
|
2020-04-08 18:08:41 +02:00 |
|
Miriam Baglioni
|
df2fc4a6d7
|
Merge remote-tracking branch 'upstream/master'
|
2020-04-08 18:07:26 +02:00 |
|
Miriam Baglioni
|
fcfef4632f
|
input parameters for country propagation preparation job
|
2020-04-08 18:07:18 +02:00 |
miconis
|
0be2e72be5
|
further implementation of tests for the deduplication of each entity. publication dump added, empty entity files created
|
2020-04-08 18:02:30 +02:00 |
|
Miriam Baglioni
|
61045e84d9
|
merged conflict in pom
|
2020-04-08 14:23:30 +02:00 |
|
Claudio Atzori
|
47f3d9b757
|
unit test for GraphHiveImporterJob
|
2020-04-08 13:24:43 +02:00 |
|
Sandro La Bruzzo
|
ba9f07a6fe
|
fixed wrong test
|
2020-04-08 13:18:20 +02:00 |
|
Miriam Baglioni
|
540da4ab61
|
new busuness logic with prepared info before actual job run
|
2020-04-08 13:04:04 +02:00 |
|
Miriam Baglioni
|
8438702b3d
|
addition in propagation constants
|
2020-04-08 10:54:01 +02:00 |
|
Miriam Baglioni
|
2afe971816
|
new implementation for country propagatio
|
2020-04-08 10:49:09 +02:00 |