 Claudio Atzori
|
44a937f4ed
|
factored out entity grouping implementation, extended to consider results from delegated authorities rather than identical records from other sources
|
2022-01-19 12:24:52 +01:00 |
|
Claudio Atzori
|
f4538f3c4c
|
cleanup
|
2021-11-19 11:33:10 +01:00 |
|
Claudio Atzori
|
2b46b87f56
|
fixed filtering criteria applied in SparkCopyRelationsNoOpenorgs to keep the parent/child relations from OpenOrgs
|
2021-11-19 11:30:29 +01:00 |
|
Claudio Atzori
|
a24b9f8268
|
[dedup] trivial refactoring
|
2021-11-18 17:12:02 +01:00 |
|
Claudio Atzori
|
c0750fb17c
|
avoid non necessary count operations over large spark datasets
|
2021-11-18 17:11:31 +01:00 |
|
Claudio Atzori
|
0a727d325d
|
[dedup] increased number of partitions in the consistency phase
|
2021-11-16 08:43:41 +01:00 |
miconis
|
611ca511db
|
set configuration property in openorgs duplicates wf
|
2021-10-07 15:39:55 +02:00 |
|
miconis
|
9646b9fd98
|
implementation of the http call for the update of openorgs suggestions
|
2021-10-07 11:29:11 +02:00 |
|
miconis
|
853333bdde
|
implementation of the whitelist for similarity relations
|
2021-09-20 16:21:47 +02:00 |
|
Claudio Atzori
|
9f4db73f30
|
updated/fixed unit tests
|
2021-08-11 15:02:51 +02:00 |
|
Claudio Atzori
|
2ee21da43b
|
suggestions from SonarLint
|
2021-08-11 12:13:22 +02:00 |
|
Claudio Atzori
|
2fff24df55
|
code formatting
|
2021-07-28 11:34:19 +02:00 |
 Sandro La Bruzzo
|
3920c69bc8
|
change implementation of resolve Relation to generate jsonRdd in output
|
2021-07-25 09:51:36 +02:00 |
|
Sandro La Bruzzo
|
058b636d4d
|
added control to check if the entity exists
|
2021-07-22 16:08:54 +02:00 |
|
Claudio Atzori
|
41b551562e
|
applying PR#115 (DatePicker) on stable_ids
|
2021-06-17 09:33:50 +02:00 |
|
Claudio Atzori
|
23b8883ab1
|
applied intellij code cleanup
|
2021-05-14 10:58:12 +02:00 |
|
Claudio Atzori
|
5afa7d3e0c
|
core utilities in dhp-common moved in external module dhp-schemas
|
2021-04-27 15:44:01 +02:00 |
|
Claudio Atzori
|
ef4bfd82e2
|
code formatting
|
2021-04-27 10:09:31 +02:00 |
|
miconis
|
6d5c14e030
|
assertions updated in entity merger test
|
2021-04-27 09:47:49 +02:00 |
|
miconis
|
3c12eeadce
|
bug fix in propagation of relations
|
2021-04-22 11:44:33 +02:00 |
|
Claudio Atzori
|
8f309b72ff
|
[dedup] using node names consistently across the workflow
|
2021-04-21 17:54:51 +02:00 |
|
Claudio Atzori
|
815b9f4d56
|
[openorgs dedup] fixed workflow parameter declarations. Introduced support for resuming the execution from intermediate steps
|
2021-04-20 17:24:45 +02:00 |
|
Claudio Atzori
|
45057440c1
|
code formatting
|
2021-04-16 17:28:25 +02:00 |
|
miconis
|
7ad573d023
|
bug fix: changed join in propagaterelations without applying filter on the id
|
2021-04-16 16:40:42 +02:00 |
|
miconis
|
f64e57c112
|
refactoring of the id generation, sparkcreatemergerels collects entities to create root id after a join
|
2021-04-15 10:59:24 +02:00 |
|
miconis
|
3525a8f504
|
id generation of representative record moved to the SparkCreateMergeRel job
|
2021-04-14 18:06:07 +02:00 |
|
miconis
|
1542196a33
|
bug fix: starting node of duplicate scan wf changed
|
2021-04-13 10:15:43 +02:00 |
|
miconis
|
369ed1cd8a
|
bug fix: lookupurl parameter added to dedup record job
|
2021-04-13 09:08:05 +02:00 |
|
Claudio Atzori
|
511c0521e5
|
[dedup] avoiding NPEs handling OpenOrg relations
|
2021-04-12 17:45:11 +02:00 |
|
miconis
|
d442e25cbc
|
bug fix: ids in self mergerels are not marked deletedbyinference=true
|
2021-04-12 15:56:22 +02:00 |
|
miconis
|
0857100fb8
|
implementation of the tests for the openorgs integration in the openaire provision
|
2021-04-07 18:42:16 +02:00 |
|
miconis
|
bf685d849f
|
addition of pids in the query for the export of openorgs for the provision, addition of ec_fields in the openorgs model
|
2021-04-07 14:27:43 +02:00 |
|
miconis
|
c39c82dfe9
|
modification of the jobs for the integration of openorgs in the provision, dedup records are no more created by merging but simply taking results of openorgs portal
|
2021-04-06 14:31:00 +02:00 |
|
Claudio Atzori
|
70e49ed53c
|
[OpenOrgsWf] trivial refactoring
|
2021-04-01 10:30:51 +02:00 |
|
Claudio Atzori
|
7941d7be29
|
WIP: using common definitions from ModelConstants
|
2021-03-31 18:33:57 +02:00 |
|
Claudio Atzori
|
72ce741ea6
|
WIP: using common definitions from ModelConstants
|
2021-03-31 17:07:13 +02:00 |
|
Claudio Atzori
|
9237d55d7f
|
[OpenOrgsWf] cleanup
|
2021-03-29 17:40:34 +02:00 |
|
miconis
|
f446580e9f
|
code refactoring (useless classes and wf removed), implementation of the test for the openorgs dedup
|
2021-03-29 16:10:46 +02:00 |
|
miconis
|
2355cc4e9b
|
minor changes and bug fix
|
2021-03-29 10:07:12 +02:00 |
|
miconis
|
28c1cdd132
|
merged stable_ids into openorgswf
|
2021-03-25 10:44:49 +01:00 |
|
miconis
|
5dfb66b0fa
|
minor changes
|
2021-03-25 10:29:34 +01:00 |
|
miconis
|
348b0ef921
|
bug fix, implementation of the workflow for the creation of raw_organizations (openorgs dedup), addition of the pid lists to the openorgs postgres db
|
2021-03-24 15:51:27 +01:00 |
|
miconis
|
98854b0124
|
minor changes
|
2021-03-19 16:57:40 +01:00 |
|
Claudio Atzori
|
972d5a3d98
|
[dedup] Datacite should be authoritative for datasets
|
2021-03-19 09:04:20 +01:00 |
|
miconis
|
1a85020572
|
bug fix in graph-mapper, changes in the implementation of the openorgs wf to create relations and populate openorgs db
|
2021-02-26 10:19:28 +01:00 |
|
miconis
|
4b2124a18e
|
implementation of the openorgs wfs, implementation of the raw_all wf to migrate openorgs db entities
|
2021-02-10 11:51:50 +01:00 |
|
miconis
|
c7e2d5a59a
|
minor changes
|
2021-01-25 12:40:45 +01:00 |
|
miconis
|
8fea29177c
|
refactoring, minor changes and implementation of the wf for openorgs with integration of organization phases into the scan wf
|
2021-01-18 16:48:08 +01:00 |
|
miconis
|
1e1aab83e3
|
implementation of the raw wf for openorgs: still not complete, some functionalities are missing
|
2020-12-21 11:58:21 +01:00 |
|
Claudio Atzori
|
d9532446eb
|
imported more diffs from master branch; code formatting
|
2020-12-10 16:14:16 +01:00 |