 Claudio Atzori
|
23b8883ab1
|
applied intellij code cleanup
|
2021-05-14 10:58:12 +02:00 |
|
Claudio Atzori
|
5afa7d3e0c
|
core utilities in dhp-common moved in external module dhp-schemas
|
2021-04-27 15:44:01 +02:00 |
|
Claudio Atzori
|
27ab8a704d
|
adjusted poms to align with the external dhp-schema module
|
2021-04-27 10:12:27 +02:00 |
|
Claudio Atzori
|
ef4bfd82e2
|
code formatting
|
2021-04-27 10:09:31 +02:00 |
|
Claudio Atzori
|
faa8f6f4e2
|
Merge branch 'stable_ids' of https://code-repo.d4science.org/D-Net/dnet-hadoop into stable_ids
|
2021-04-27 09:57:03 +02:00 |
miconis
|
6d5c14e030
|
assertions updated in entity merger test
|
2021-04-27 09:47:49 +02:00 |
|
Claudio Atzori
|
c2bb03c8b5
|
depending on external dhp-schemas module
|
2021-04-23 17:57:35 +02:00 |
|
miconis
|
3c12eeadce
|
bug fix in propagation of relations
|
2021-04-22 11:44:33 +02:00 |
|
Claudio Atzori
|
8f309b72ff
|
[dedup] using node names consistently across the workflow
|
2021-04-21 17:54:51 +02:00 |
|
Claudio Atzori
|
815b9f4d56
|
[openorgs dedup] fixed workflow parameter declarations. Introduced support for resuming the execution from intermediate steps
|
2021-04-20 17:24:45 +02:00 |
|
Claudio Atzori
|
45057440c1
|
code formatting
|
2021-04-16 17:28:25 +02:00 |
|
miconis
|
7ad573d023
|
bug fix: changed join in propagaterelations without applying filter on the id
|
2021-04-16 16:40:42 +02:00 |
|
miconis
|
f64e57c112
|
refactoring of the id generation, sparkcreatemergerels collects entities to create root id after a join
|
2021-04-15 10:59:24 +02:00 |
|
miconis
|
3525a8f504
|
id generation of representative record moved to the SparkCreateMergeRel job
|
2021-04-14 18:06:07 +02:00 |
|
miconis
|
1542196a33
|
bug fix: starting node of duplicate scan wf changed
|
2021-04-13 10:15:43 +02:00 |
|
miconis
|
369ed1cd8a
|
bug fix: lookupurl parameter added to dedup record job
|
2021-04-13 09:08:05 +02:00 |
|
Claudio Atzori
|
511c0521e5
|
[dedup] avoiding NPEs handling OpenOrg relations
|
2021-04-12 17:45:11 +02:00 |
|
miconis
|
d442e25cbc
|
bug fix: ids in self mergerels are not marked deletedbyinference=true
|
2021-04-12 15:56:22 +02:00 |
|
miconis
|
0857100fb8
|
implementation of the tests for the openorgs integration in the openaire provision
|
2021-04-07 18:42:16 +02:00 |
|
miconis
|
bf685d849f
|
addition of pids in the query for the export of openorgs for the provision, addition of ec_fields in the openorgs model
|
2021-04-07 14:27:43 +02:00 |
|
miconis
|
c39c82dfe9
|
modification of the jobs for the integration of openorgs in the provision, dedup records are no more created by merging but simply taking results of openorgs portal
|
2021-04-06 14:31:00 +02:00 |
|
Claudio Atzori
|
70e49ed53c
|
[OpenOrgsWf] trivial refactoring
|
2021-04-01 10:30:51 +02:00 |
|
Claudio Atzori
|
7941d7be29
|
WIP: using common definitions from ModelConstants
|
2021-03-31 18:33:57 +02:00 |
|
Claudio Atzori
|
72ce741ea6
|
WIP: using common definitions from ModelConstants
|
2021-03-31 17:07:13 +02:00 |
|
Claudio Atzori
|
9237d55d7f
|
[OpenOrgsWf] cleanup
|
2021-03-29 17:40:34 +02:00 |
|
miconis
|
f446580e9f
|
code refactoring (useless classes and wf removed), implementation of the test for the openorgs dedup
|
2021-03-29 16:10:46 +02:00 |
|
miconis
|
2355cc4e9b
|
minor changes and bug fix
|
2021-03-29 10:07:12 +02:00 |
|
miconis
|
28c1cdd132
|
merged stable_ids into openorgswf
|
2021-03-25 10:44:49 +01:00 |
|
miconis
|
5dfb66b0fa
|
minor changes
|
2021-03-25 10:29:34 +01:00 |
|
miconis
|
348b0ef921
|
bug fix, implementation of the workflow for the creation of raw_organizations (openorgs dedup), addition of the pid lists to the openorgs postgres db
|
2021-03-24 15:51:27 +01:00 |
|
miconis
|
98854b0124
|
minor changes
|
2021-03-19 16:57:40 +01:00 |
|
Claudio Atzori
|
972d5a3d98
|
[dedup] Datacite should be authoritative for datasets
|
2021-03-19 09:04:20 +01:00 |
|
miconis
|
1a85020572
|
bug fix in graph-mapper, changes in the implementation of the openorgs wf to create relations and populate openorgs db
|
2021-02-26 10:19:28 +01:00 |
|
miconis
|
4b2124a18e
|
implementation of the openorgs wfs, implementation of the raw_all wf to migrate openorgs db entities
|
2021-02-10 11:51:50 +01:00 |
|
miconis
|
c7e2d5a59a
|
minor changes
|
2021-01-25 12:40:45 +01:00 |
|
miconis
|
8fea29177c
|
refactoring, minor changes and implementation of the wf for openorgs with integration of organization phases into the scan wf
|
2021-01-18 16:48:08 +01:00 |
|
miconis
|
1e1aab83e3
|
implementation of the raw wf for openorgs: still not complete, some functionalities are missing
|
2020-12-21 11:58:21 +01:00 |
|
Claudio Atzori
|
d9532446eb
|
imported more diffs from master branch; code formatting
|
2020-12-10 16:14:16 +01:00 |
|
Claudio Atzori
|
1eaad89a3c
|
do not fail on uknown properties when grouping entities by ID
|
2020-12-10 15:56:11 +01:00 |
|
Claudio Atzori
|
758d27745d
|
cleaning tab characters from text fields
|
2020-11-27 16:07:24 +01:00 |
|
Claudio Atzori
|
5151850a19
|
CROSSREF and DATACITE constants moved in common ModelConstants
|
2020-11-26 13:08:36 +01:00 |
|
Claudio Atzori
|
d0d5525d40
|
minor changes
|
2020-11-26 11:04:17 +01:00 |
|
Claudio Atzori
|
13eae4b31e
|
GroupEntitiesSparkJob must read all graph paths but relations
|
2020-11-26 11:04:01 +01:00 |
|
Claudio Atzori
|
76363a8512
|
SimpleDateFormat is not thread safe; improved error reporting in case of invalid dates
|
2020-11-26 11:03:12 +01:00 |
|
Claudio Atzori
|
e208b03755
|
renamed workflow
|
2020-11-25 14:55:50 +01:00 |
|
Claudio Atzori
|
dfd6205b95
|
Consistency graph workflow merges all the entities by ID
|
2020-11-25 14:55:32 +01:00 |
|
Claudio Atzori
|
e5da4ee9b1
|
dedup workflow using the common PidComparator
|
2020-11-04 15:02:02 +01:00 |
|
Claudio Atzori
|
385214eeae
|
code formatting
|
2020-10-30 15:47:05 +01:00 |
|
miconis
|
c4a59d1b9a
|
merge with the master to port the new packages
|
2020-10-20 16:07:30 +02:00 |
|
miconis
|
708d887e64
|
minor changes
|
2020-10-20 15:12:19 +02:00 |