 Miriam Baglioni
|
e2dfe8b656
|
removed not used action
|
2020-04-18 14:16:07 +02:00 |
|
Miriam Baglioni
|
437ebbad76
|
refactorign
|
2020-04-18 14:15:09 +02:00 |
|
Miriam Baglioni
|
9a8876ac86
|
added needed parameter
|
2020-04-18 14:14:08 +02:00 |
|
Miriam Baglioni
|
9854852878
|
refactoring
|
2020-04-18 14:13:16 +02:00 |
|
Miriam Baglioni
|
890ec28f0f
|
input parameters for preparation step1
|
2020-04-18 14:09:37 +02:00 |
|
Miriam Baglioni
|
fbf5c27c27
|
Added preparation classes before actual propagation
|
2020-04-18 14:09:03 +02:00 |
|
Miriam Baglioni
|
72c63a326e
|
removed unuseful class
|
2020-04-17 17:14:51 +02:00 |
|
Miriam Baglioni
|
00c2ca3ee5
|
-
|
2020-04-17 17:14:25 +02:00 |
|
Miriam Baglioni
|
5cd092114f
|
use mergeFrom method to add the new community contexts
|
2020-04-17 17:13:18 +02:00 |
|
Miriam Baglioni
|
264c82f21e
|
minor
|
2020-04-17 16:54:46 +02:00 |
|
Miriam Baglioni
|
8c079c7a49
|
unit test for orcid to result propagation from semrel
|
2020-04-17 16:53:03 +02:00 |
|
Miriam Baglioni
|
eacd140a98
|
added missing parameter(s)
|
2020-04-17 16:52:30 +02:00 |
|
Miriam Baglioni
|
390e250faf
|
use the addPid method of the Author class to add a new pid
|
2020-04-17 16:52:02 +02:00 |
|
Miriam Baglioni
|
b46b080ddc
|
use mergeFrom method call to add the country(ies) instead of modify the result directly.
|
2020-04-17 16:50:54 +02:00 |
|
Miriam Baglioni
|
c4987dd12a
|
minor
|
2020-04-17 16:49:08 +02:00 |
|
Miriam Baglioni
|
5d772e5263
|
new implementation of propagation of community to result from organization that exploits the prepared info
|
2020-04-16 18:45:22 +02:00 |
|
Miriam Baglioni
|
fff1e5ec39
|
classes to (de)serialize the data provided in the preparation step
|
2020-04-16 18:44:43 +02:00 |
|
Miriam Baglioni
|
3fd9d6b02f
|
preparation phase for the propagation of community to result from organization
|
2020-04-16 18:43:55 +02:00 |
|
Miriam Baglioni
|
a9120164aa
|
added hive parameter and a step of reset of the working dir in the workflow
|
2020-04-16 18:42:04 +02:00 |
|
Miriam Baglioni
|
6afbd542ca
|
changed the save mode to avoid NegativeArraySize... error. Needed to modify also the preparationstep2
|
2020-04-16 18:40:14 +02:00 |
|
Miriam Baglioni
|
d60fd36046
|
changed the save method
|
2020-04-16 16:14:15 +02:00 |
|
Miriam Baglioni
|
951b13ac46
|
input parameters and workflow for new implementation of propagation of orcid to result from semrel and preparation phases
|
2020-04-16 16:13:10 +02:00 |
|
Miriam Baglioni
|
4d89f3dfed
|
removed unuseful classes
|
2020-04-16 16:11:44 +02:00 |
|
Miriam Baglioni
|
5e72a51f11
|
-
|
2020-04-16 16:11:20 +02:00 |
|
Miriam Baglioni
|
c33a593381
|
renamed
|
2020-04-16 16:09:47 +02:00 |
|
Miriam Baglioni
|
0e5399bf74
|
seconf phase of data preparation. Groups all the possible updates by id
|
2020-04-16 16:08:51 +02:00 |
|
Miriam Baglioni
|
548ba915ac
|
first phase of data preparation. For each result type (parallel) it produces the possible updates
|
2020-04-16 15:58:42 +02:00 |
|
Miriam Baglioni
|
243013cea3
|
to (de)serialize the association from the resultId and the list of autoritative authors with orcid to possibly propagate
|
2020-04-16 15:57:29 +02:00 |
|
Miriam Baglioni
|
ac3ad25b36
|
to (de)serialize needed information of the author to determine if the orcid can be passed (name, surname, fullname (?), orcid)
|
2020-04-16 15:56:33 +02:00 |
|
Miriam Baglioni
|
d6cd700a32
|
new implementation that exploits prepared information (the list of possible updates: resultId - possible list of orcid to be added
|
2020-04-16 15:55:25 +02:00 |
|
Miriam Baglioni
|
f077f22f73
|
minor
|
2020-04-16 15:54:16 +02:00 |
|
Miriam Baglioni
|
fd5d792e35
|
refactoring
|
2020-04-16 15:53:34 +02:00 |
|
Miriam Baglioni
|
08227cfcbd
|
resources needed for running the test on propagation of result to organization from institutional repositories
|
2020-04-16 11:06:10 +02:00 |
|
Miriam Baglioni
|
a97e915c24
|
test unit for propagation of result to organization from institutional repository
|
2020-04-16 11:05:21 +02:00 |
|
Miriam Baglioni
|
b078710924
|
modification to the test due to the removal of unused parameters
|
2020-04-16 11:04:39 +02:00 |
|
Miriam Baglioni
|
a5e5c81a2c
|
input parameters and workflow definition for propagation of result to organization from institutional repositories
|
2020-04-16 11:03:41 +02:00 |
|
Miriam Baglioni
|
5e1bd67680
|
removed unuseful parameter
|
2020-04-16 11:02:01 +02:00 |
|
Miriam Baglioni
|
eaf19ce01b
|
removed unuseful class
|
2020-04-16 10:59:33 +02:00 |
|
Miriam Baglioni
|
7bd49abbef
|
commit to delete
|
2020-04-16 10:59:09 +02:00 |
|
Miriam Baglioni
|
53f418098b
|
added the isTest checkpoint
|
2020-04-16 10:53:48 +02:00 |
|
Miriam Baglioni
|
c28333d43f
|
minor
|
2020-04-16 10:52:50 +02:00 |
|
Miriam Baglioni
|
a8100baed6
|
changed the way to save the results to aviod NegativeArray... error
|
2020-04-16 10:50:09 +02:00 |
|
Miriam Baglioni
|
79b978ec57
|
refactoring
|
2020-04-16 10:48:41 +02:00 |
|
Miriam Baglioni
|
3577219127
|
removed unuseful classes
|
2020-04-15 12:45:49 +02:00 |
|
Miriam Baglioni
|
964b22d418
|
modified the writing of the new relations. before: read old rels, add the new ones to them, write all the relations in new location. Now: first step of the wf copies the old relation i new location. If new relations are found, they are saved in the new location in append mode.
|
2020-04-15 12:32:01 +02:00 |
|
Miriam Baglioni
|
43f0590d4b
|
change in the testing because the business logic is changed.
|
2020-04-15 12:29:50 +02:00 |
|
Miriam Baglioni
|
473d17767c
|
new business logic for the actual propagation. It exploits previously computed information
|
2020-04-15 12:25:44 +02:00 |
|
Miriam Baglioni
|
6a377a7582
|
class to compute some information needed for the actual propagation
|
2020-04-15 12:25:11 +02:00 |
|
Miriam Baglioni
|
5a3487280d
|
classes to serialize/deserialize the prepared data
|
2020-04-15 12:24:36 +02:00 |
|
Miriam Baglioni
|
62b09be43c
|
added correct descritption for parameter isSparkSessionManaged
|
2020-04-15 12:23:06 +02:00 |
|
Miriam Baglioni
|
1859ce8902
|
minor refactoring
|
2020-04-15 12:21:31 +02:00 |
|
Miriam Baglioni
|
27f1d3ee8f
|
minor refactoring
|
2020-04-15 12:21:05 +02:00 |
|
Miriam Baglioni
|
3f4b579e7f
|
new workflow. It is composed of four steps. The first removes the directory where to store the results. The second copies the relation to the new locatio, the third id the preparation phase and then the actual propagation
|
2020-04-14 16:49:24 +02:00 |
|
Miriam Baglioni
|
ca2b40952e
|
minor changes
|
2020-04-14 16:48:02 +02:00 |
|
Miriam Baglioni
|
61d39e659e
|
parameters for the project2result propagation phase
|
2020-04-14 16:47:39 +02:00 |
|
Miriam Baglioni
|
92f19fa0a0
|
parameters for the project2result preparation phase
|
2020-04-14 16:46:57 +02:00 |
|
Miriam Baglioni
|
cadab9b81d
|
new implementation for result to project propagation. Use the prepared info in propagation
|
2020-04-14 16:46:07 +02:00 |
|
Miriam Baglioni
|
ceb1f299bf
|
minor changes
|
2020-04-14 16:45:12 +02:00 |
|
Miriam Baglioni
|
e0038bde5b
|
Support class to serialize/deserialize the association project, set of linked results
|
2020-04-14 15:32:12 +02:00 |
|
Miriam Baglioni
|
c0bebb7c35
|
code to compute the prepared information used in the actual propagation step. This step will produce who files: one with potential updates (association between projects and a list of results), the other already linked entities (association between projects and the list of results already linked to them)
|
2020-04-14 15:31:26 +02:00 |
|
Miriam Baglioni
|
f47ee5b78e
|
directory where to store the prepared infor before actual propagation will take place
|
2020-04-14 15:29:21 +02:00 |
|
Miriam Baglioni
|
36cc9516d8
|
the starting relation set for testing
|
2020-04-14 15:28:34 +02:00 |
|
Miriam Baglioni
|
4b01dc60e6
|
test unit for result to project propagation
|
2020-04-14 15:28:00 +02:00 |
|
Miriam Baglioni
|
8f12292daa
|
changed the way to save the results on filesystem
|
2020-04-11 16:47:34 +02:00 |
|
Miriam Baglioni
|
87f802821e
|
new workflow for country propagation: it is composed of the preparation step and in the propagation. The propagation part runs in parallel on the result types
|
2020-04-11 16:40:22 +02:00 |
|
Miriam Baglioni
|
a562080b0b
|
parameters to be used in the prepared Job and in the actual country propagation job
|
2020-04-11 16:39:17 +02:00 |
|
Miriam Baglioni
|
1251ad4455
|
removed unuseful class
|
2020-04-11 16:38:13 +02:00 |
|
Miriam Baglioni
|
aef9b3aa90
|
new parametric implementation of country propagation. Exploits information compute before and broadcasts it to each executor
|
2020-04-11 16:36:59 +02:00 |
|
Miriam Baglioni
|
a2d833d5dd
|
step of data preparation before actual country propagation will take palce
|
2020-04-11 16:36:03 +02:00 |
|
Miriam Baglioni
|
6897c920a2
|
classes in support of new implementation of country propagation
|
2020-04-11 16:35:26 +02:00 |
|
Miriam Baglioni
|
85766a02d8
|
added dependency to use hive on local machine
|
2020-04-11 16:34:22 +02:00 |
|
Miriam Baglioni
|
79b8ea4fed
|
prepared information to be used in actual country propagation. Subset of info
|
2020-04-11 16:29:41 +02:00 |
|
Miriam Baglioni
|
1822476613
|
Test for country propagation
|
2020-04-11 16:28:09 +02:00 |
|
Miriam Baglioni
|
7783b09c5b
|
new implementation for result to project propagation. Prepare some info to be used in propagation
|
2020-04-11 16:26:23 +02:00 |
|
Miriam Baglioni
|
90469789b9
|
two new classes fro new implementation of project to result propagation
|
2020-04-09 13:29:01 +02:00 |
|
Miriam Baglioni
|
627ad58a8b
|
new wf definition
|
2020-04-09 11:33:19 +02:00 |
|
Miriam Baglioni
|
9c63c4840d
|
new workflow and parameters for country propagation
|
2020-04-08 19:13:42 +02:00 |
|
Miriam Baglioni
|
a2d309545b
|
new parametrized implementation for country propagation
|
2020-04-08 19:12:59 +02:00 |
|
Miriam Baglioni
|
6dfdba9ef7
|
new parametrized implementation for country propagation
|
2020-04-08 18:14:37 +02:00 |
|
Miriam Baglioni
|
03f7cb6402
|
new parametrized implementation for country propagation
|
2020-04-08 18:08:41 +02:00 |
|
Miriam Baglioni
|
fcfef4632f
|
input parameters for country propagation preparation job
|
2020-04-08 18:07:18 +02:00 |
|
Miriam Baglioni
|
540da4ab61
|
new busuness logic with prepared info before actual job run
|
2020-04-08 13:04:04 +02:00 |
|
Miriam Baglioni
|
8438702b3d
|
addition in propagation constants
|
2020-04-08 10:54:01 +02:00 |
|
Miriam Baglioni
|
2afe971816
|
new implementation for country propagatio
|
2020-04-08 10:49:09 +02:00 |
|
Miriam Baglioni
|
beebbcf66b
|
new config for countrypropagation
|
2020-04-08 10:31:29 +02:00 |
|
Miriam Baglioni
|
dd011f4a95
|
to make them visible to Claudio
|
2020-03-30 10:55:47 +02:00 |
|
Miriam Baglioni
|
b1af90a45f
|
to make it visible to Claudio
|
2020-03-30 10:50:03 +02:00 |
|
Miriam Baglioni
|
19d7f8b51d
|
decommented execution for some of the result type for testing purposes
|
2020-03-24 16:49:46 +01:00 |
|
Miriam Baglioni
|
ad24c8478f
|
added missing parameter
|
2020-03-24 16:19:59 +01:00 |
|
Miriam Baglioni
|
46094a3eec
|
bug fixing for implementation with dataset
|
2020-03-24 16:19:36 +01:00 |
|
Miriam Baglioni
|
ad712f2d79
|
added the needed variables in the config and read the variables in the workflow
|
2020-03-23 17:11:36 +01:00 |
|
Miriam Baglioni
|
f1e9fe9752
|
changed implementation using dataset and query on hive
|
2020-03-23 17:11:00 +01:00 |
|
Miriam Baglioni
|
f09cd1e911
|
removed unuseful variable in the configuration
|
2020-03-23 17:10:14 +01:00 |
|
Miriam Baglioni
|
9418e3d4fa
|
read dataset from files instead of using hive tables
|
2020-03-23 17:09:27 +01:00 |
|
Miriam Baglioni
|
a7bf037306
|
remove unused class
|
2020-03-23 14:36:43 +01:00 |
|
Miriam Baglioni
|
8ab8b6b0bf
|
minor
|
2020-03-23 14:35:23 +01:00 |
|
Miriam Baglioni
|
30d58fd98c
|
change the configuration of the workflow
|
2020-03-23 14:32:49 +01:00 |
|
Miriam Baglioni
|
a440152b46
|
refactoring
|
2020-03-23 14:30:56 +01:00 |
|
Miriam Baglioni
|
47561f3597
|
changed the implementation from rdd to dataset got from sql queries (on hive)
|
2020-03-23 11:58:32 +01:00 |
|
Miriam Baglioni
|
67ea3cf3ed
|
changed the way to read the file with info on resource or relation. From sequenceFile to textFile
|
2020-03-17 16:32:05 +01:00 |