 Miriam Baglioni
|
14eda4f46e
|
added method to try to put inputstream to zenodo
|
2020-08-05 14:18:25 +02:00 |
|
Miriam Baglioni
|
e737a47270
|
added classes to try to send input stream to zenodo for the upload
|
2020-08-05 14:17:40 +02:00 |
|
Miriam Baglioni
|
873e9cd50c
|
changed hadoop setting to connect to s3
|
2020-08-04 15:37:25 +02:00 |
|
Miriam Baglioni
|
5b651abf82
|
merge branch with master
|
2020-08-04 10:14:07 +02:00 |
|
Miriam Baglioni
|
aa9f3d9698
|
changed logic for save in s3 directly
|
2020-08-03 18:06:18 +02:00 |
|
Miriam Baglioni
|
d465f0eec9
|
added fulltext to result
|
2020-08-03 18:03:27 +02:00 |
|
Miriam Baglioni
|
c892c7dfa7
|
changed to query for community map just once and save the result for remaining executions
|
2020-08-03 17:56:31 +02:00 |
 Michele Artini
|
652b13abb6
|
Merge branch 'master' into nsprefix_blacklist
|
2020-07-31 07:58:37 +02:00 |
 Claudio Atzori
|
cd631bb5bc
|
defaults fixed in the cleaning workflow forces result.publisher to NULL when result.publisher.value in empty
|
2020-07-30 17:03:53 +02:00 |
|
Miriam Baglioni
|
57c87b7653
|
re-implemented to fix issue on not serializable Set<String> variable
|
2020-07-30 16:43:43 +02:00 |
|
Miriam Baglioni
|
ef8e5957b5
|
added specific directory where to save results
|
2020-07-30 16:42:46 +02:00 |
|
Miriam Baglioni
|
75f3361c85
|
-
|
2020-07-30 16:41:31 +02:00 |
|
Miriam Baglioni
|
3f695b25fa
|
refactoring
|
2020-07-30 16:40:15 +02:00 |
|
Miriam Baglioni
|
e623f12bef
|
refactoring
|
2020-07-30 16:32:59 +02:00 |
|
Miriam Baglioni
|
ff7d05abb4
|
added support class to store the couple organizationId representativeId gaot from sql query on hive
|
2020-07-30 16:32:04 +02:00 |
|
Miriam Baglioni
|
cf6d80b2ab
|
added command to close the writer
|
2020-07-30 16:31:22 +02:00 |
|
Miriam Baglioni
|
f985bca37b
|
added USER_CLAIM constant value
|
2020-07-30 16:25:26 +02:00 |
|
Claudio Atzori
|
4bbfcf1ac6
|
Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop
|
2020-07-30 16:25:06 +02:00 |
|
Claudio Atzori
|
4ff8007518
|
added function to set the missing vocabulary names, used in the cleaning workflow as a pre-cleaning step
|
2020-07-30 16:24:39 +02:00 |
|
Michele Artini
|
bdece15ca0
|
blacklist of nsprefix
|
2020-07-30 16:13:38 +02:00 |
Sandro La Bruzzo
|
c97c8f0c44
|
implemented new oozie job to extract entities in a separate dataset
|
2020-07-30 12:13:58 +02:00 |
 Sandro La Bruzzo
|
3010a362bc
|
updated changing in the workflow of provision in the phase of aggregation. Removed serialization in JSON RDD and used spark Dataset
|
2020-07-30 09:25:56 +02:00 |
|
Sandro La Bruzzo
|
16ae3c9ccf
|
updated changing in the workflow of provision in the phase of aggregation. Removed serialization in JSON RDD and used spark Dataset
|
2020-07-30 09:25:32 +02:00 |
|
Miriam Baglioni
|
76bcab98ce
|
added code to filter out null originalId from the dump
|
2020-07-29 18:28:21 +02:00 |
|
Miriam Baglioni
|
86bab79512
|
-
|
2020-07-29 18:20:22 +02:00 |
|
Miriam Baglioni
|
31791dcf3d
|
fixed wrong property file path name
|
2020-07-29 18:20:08 +02:00 |
|
Miriam Baglioni
|
9e722aa1ef
|
-
|
2020-07-29 18:00:08 +02:00 |
|
Miriam Baglioni
|
d22f106f27
|
added constant to identify datasource associated to funders
|
2020-07-29 17:56:55 +02:00 |
|
Miriam Baglioni
|
40e194fe2f
|
added check to not dump datasources related to funders
|
2020-07-29 17:56:18 +02:00 |
|
Miriam Baglioni
|
074e9ab75e
|
refactoring
|
2020-07-29 17:42:50 +02:00 |
|
Miriam Baglioni
|
8ad8dac7d4
|
merge branch with fork master
|
2020-07-29 17:38:28 +02:00 |
|
Miriam Baglioni
|
9fa82dc93b
|
fixed issue
|
2020-07-29 17:36:16 +02:00 |
|
Miriam Baglioni
|
8907648d6a
|
-
|
2020-07-29 17:35:47 +02:00 |
|
Miriam Baglioni
|
6d0f08277b
|
classes to implement the dump of the whole graph.
|
2020-07-29 17:03:19 +02:00 |
|
Miriam Baglioni
|
b5f995ab12
|
refactoring
|
2020-07-29 16:59:48 +02:00 |
|
Miriam Baglioni
|
f7a87cc447
|
added new constants value
|
2020-07-29 16:58:40 +02:00 |
|
Miriam Baglioni
|
b71d12cf26
|
refactoring
|
2020-07-29 16:52:44 +02:00 |
|
Miriam Baglioni
|
a8d65b68cb
|
changed to delete the part to check if it was a test or a real execution
|
2020-07-29 16:47:57 +02:00 |
|
Miriam Baglioni
|
3ec2392904
|
Added new class to move the place the split is effectively run
|
2020-07-29 16:46:50 +02:00 |
|
Miriam Baglioni
|
6c2223d1fc
|
added code to get the openaire id for contexts
|
2020-07-24 17:30:15 +02:00 |
|
Miriam Baglioni
|
afd54c1684
|
removed not needed upload and refactoring
|
2020-07-24 17:28:56 +02:00 |
|
Miriam Baglioni
|
7b0569d989
|
changed to map also the result associated to the whole graph
|
2020-07-24 17:28:11 +02:00 |
|
Miriam Baglioni
|
082225ad61
|
-
|
2020-07-24 17:27:26 +02:00 |
|
Miriam Baglioni
|
968c59d97a
|
added teh logic to dump also the products for the whole graph. They will miss collected from and context information that will be materialized as new relations
|
2020-07-24 17:25:19 +02:00 |
|
Miriam Baglioni
|
332258d199
|
split the classes related to the communities dump and to the whole graph dump
|
2020-07-24 17:21:48 +02:00 |
|
Claudio Atzori
|
56bbfdc65d
|
introduced parameter 'numParitions', driving the hive DB table data partitioning. Currently specified only for table 'project'
|
2020-07-23 08:54:10 +02:00 |
|
Claudio Atzori
|
ebf60020ac
|
map results as OPRs in case of missing //CobjCategory/@type and the vocabulary dnet:result_typologies doesn't resolve the super type
|
2020-07-20 19:01:10 +02:00 |
|
Miriam Baglioni
|
355d7e426e
|
added dumo for project - not finished
|
2020-07-20 18:54:43 +02:00 |
|
Miriam Baglioni
|
a2f01e5259
|
added getter and setter
|
2020-07-20 18:54:17 +02:00 |
|
Miriam Baglioni
|
40bbe94f7c
|
merge with master fork
|
2020-07-20 18:10:03 +02:00 |