 Michele Artini
|
3268570b2c
|
mapping of project PIDs
|
2024-02-22 14:47:21 +01:00 |
 Claudio Atzori
|
42f5506306
|
[orcid enrichment] fixed directory cleanup before distcp
|
2024-02-05 09:45:36 +02:00 |
|
Claudio Atzori
|
2655eea5bc
|
[orcid enrichment] drop paths before copying the non-modifyed contents
|
2024-01-19 16:28:05 +01:00 |
 Miriam Baglioni
|
e711a05229
|
fixed conflicts
|
2024-01-10 11:03:42 +01:00 |
|
Claudio Atzori
|
62104790ae
|
added metaresourcetype to the result hive DB view
|
2023-12-21 12:27:10 +01:00 |
|
Miriam Baglioni
|
4740c808f7
|
-
|
2023-12-20 14:26:54 +01:00 |
|
Claudio Atzori
|
259c69e446
|
[orcid enrichment] fixed workflow definition
|
2023-12-06 19:41:53 +01:00 |
|
Claudio Atzori
|
2a233a89aa
|
[graph grouping] added isLookupUrl to the workflow definition, passed to the grouping spark aciton
|
2023-12-03 13:32:52 +01:00 |
Sandro La Bruzzo
|
f718caaac9
|
Added copy of the untouched entities of the graph
|
2023-11-30 14:51:00 +01:00 |
|
Sandro La Bruzzo
|
7b5e04f37e
|
removed Orcid intersection on DOIBoost
|
2023-11-30 14:36:50 +01:00 |
|
Sandro La Bruzzo
|
34a4b3cbdf
|
Implemented ORCID Enrichment
|
2023-11-24 12:39:58 +01:00 |
|
Claudio Atzori
|
7fc621cdec
|
added defaults to the graph resolution workflow config-default.xml
|
2023-10-20 22:28:12 +02:00 |
 Giambattista Bloisi
|
6cc7d8ca7b
|
GroupEntities and DispatchEntites are now merged in GroupEntitiesSparkJob
|
2023-08-30 10:43:31 +02:00 |
|
Giambattista Bloisi
|
95cd2b9b1e
|
Make filterInvisible a mandatory parameter of DispathEntitiesSparkJob
Make filterInvisible a mandatory parameter of both dedup/consistency and graph/group oozie workflows
|
2023-08-10 11:53:48 +02:00 |
|
Giambattista Bloisi
|
fab9920271
|
DispatchEntitiesSparkJob: manage all entity types together, support filtering by dataInfo.invisible flag
|
2023-08-09 15:41:43 +02:00 |
|
Claudio Atzori
|
b76a47b103
|
[aggregator graph] added column alias when mapping organization PIDs from the OpenOrgs database
|
2023-06-13 11:38:10 +02:00 |
|
Claudio Atzori
|
e1409ffe80
|
update sql query to return distinct pids
|
2023-06-12 09:47:45 +02:00 |
|
Claudio Atzori
|
851f664bd9
|
Merge branch 'beta' into graph_cleaning_refactoring
|
2023-05-02 09:55:40 +02:00 |
|
Claudio Atzori
|
a2dcb06daf
|
added eoscifguidelines in the result view; removed compute statistics statements
|
2023-04-11 10:43:32 +02:00 |
|
Claudio Atzori
|
b502f86523
|
fixed input path supplemented to GetDatasourceFromCountry; adjusted the various spark.sql.shuffle.partitions
|
2023-03-24 13:09:12 +01:00 |
|
Claudio Atzori
|
c07857fa37
|
[graph cleaning] unit tests & cleanup
|
2023-03-23 15:57:47 +01:00 |
|
Claudio Atzori
|
4f5ba0ed52
|
[graph cleaning] WIP: refactoring of the cleaning stages, unit tests
|
2023-03-21 14:41:20 +01:00 |
|
Claudio Atzori
|
6d3d18d8b5
|
[graph cleaning] WIP: refactoring of the cleaning stages
|
2023-03-16 17:23:36 +01:00 |
|
Claudio Atzori
|
e28d395e87
|
[aggregator graph] using dedicated path to sync claims, adjusted paths with wildcards
|
2023-03-08 21:16:52 +01:00 |
|
Claudio Atzori
|
5b8fd37314
|
[aggregator graph] using dedicated path to sync claims
|
2023-03-08 15:28:14 +01:00 |
|
Claudio Atzori
|
8ec0d62d91
|
pre-group the records in each table before joning the contents from BETA and PROD together
|
2023-03-02 14:49:19 +01:00 |
|
Claudio Atzori
|
9c59dac859
|
followup changes reorganising the mdstore synchronisation mechanism
|
2023-03-01 10:16:20 +01:00 |
|
Sandro La Bruzzo
|
78e51c182a
|
Added missing parametero to raw all workflow
|
2023-02-28 10:16:01 +01:00 |
|
Michele Artini
|
fddcf701e9
|
updated the order of the compatibilities
|
2023-02-22 12:07:09 +01:00 |
|
Sandro La Bruzzo
|
6c81a161d2
|
Merge remote-tracking branch 'origin/beta' into 8231-mdstore-synch-improve
|
2023-02-08 10:29:09 +01:00 |
|
Miriam Baglioni
|
a485a94956
|
[Cleaning] fixed parameter name in property file
|
2022-12-08 16:59:34 +01:00 |
|
Miriam Baglioni
|
3d99b78d94
|
[Cleaning] fixed error in parameter (workingPath to workingDir)
|
2022-12-08 10:25:02 +01:00 |
|
Sandro La Bruzzo
|
5a48a2fb18
|
implemented synch for single mdstore
|
2022-12-01 11:34:43 +01:00 |
|
Claudio Atzori
|
58c05731f9
|
[graph cleaning] WIP: testing the collectedfron and hostedby patch procedure
|
2022-11-29 11:21:51 +01:00 |
|
Claudio Atzori
|
24ef301cc1
|
[graph cleaning] patch the result's collectedfrom and hostedby identifiers according to the datasource master-duplicate mapping
|
2022-11-28 09:54:18 +01:00 |
|
Claudio Atzori
|
ae7cd0735a
|
[graph2hive] more partitions
|
2022-10-14 15:47:58 +02:00 |
|
Claudio Atzori
|
6163ecbf63
|
[cleaning] renamed parameters in wf action
|
2022-10-11 11:20:03 +02:00 |
|
Claudio Atzori
|
b301e9fdff
|
[cleaning] renamed action name/description
|
2022-10-11 11:08:52 +02:00 |
|
Claudio Atzori
|
8d97949316
|
[cleaning] fixed loop in wf nodes
|
2022-10-07 09:52:45 +02:00 |
|
Claudio Atzori
|
e45ec15221
|
Merge branch 'beta' into clean_country
|
2022-09-19 11:34:02 +02:00 |
|
Claudio Atzori
|
192215a18e
|
merged from branch discard-non-wellformed
|
2022-09-19 10:17:10 +02:00 |
|
Claudio Atzori
|
e370e940d8
|
[aggregator graph] save invalid records aside for further inspection
|
2022-09-16 14:06:28 +02:00 |
|
Claudio Atzori
|
a0919ed495
|
[aggregator graph] save invalid records aside for further inspection
|
2022-09-14 13:27:39 +02:00 |
|
Claudio Atzori
|
14dc909a14
|
Merge branch 'beta' into clean_country
|
2022-09-09 10:38:17 +02:00 |
|
Claudio Atzori
|
cb7c07c54e
|
[scholix] added step to create tar archive
|
2022-08-11 11:25:24 +02:00 |
|
Claudio Atzori
|
2aa16d0432
|
[scholix] fixed OpenCitation dump procedure
|
2022-08-10 17:39:29 +02:00 |
|
Miriam Baglioni
|
390013a4b2
|
mergin with branch beta
|
2022-08-08 12:30:31 +02:00 |
|
Miriam Baglioni
|
a7a18d7630
|
[Graph Dump] removed code for the dump from the project. Fixed issues in tests when possible
|
2022-08-04 17:40:40 +02:00 |
|
Claudio Atzori
|
09ccc7b472
|
Merge branch 'beta' into project_organization_contribution
|
2022-07-28 09:49:59 +02:00 |
|
Miriam Baglioni
|
5968ec018d
|
[Clean Country] modified workflow and added param file
|
2022-07-22 16:48:38 +02:00 |