Claudio Atzori
|
09ccc7b472
|
Merge branch 'beta' into project_organization_contribution
|
2022-07-28 09:49:59 +02:00 |
Claudio Atzori
|
0c1cfee396
|
mapping oaf:fulltext elements in the result.fulltext field
|
2022-07-11 17:34:59 +02:00 |
Claudio Atzori
|
7da24c1dec
|
added more logging
|
2022-06-28 13:47:49 +02:00 |
Claudio Atzori
|
5130eac247
|
mapping by participant project contribution
|
2022-06-24 17:16:42 +02:00 |
Claudio Atzori
|
4c8e820ff0
|
mapping relationship from trasformed records based on oaf:relation
|
2022-06-14 08:49:02 +02:00 |
Claudio Atzori
|
116902c028
|
mapping relationship from trasformed records based on oaf:relation
|
2022-06-13 14:31:48 +02:00 |
Claudio Atzori
|
77bc9863e9
|
[openorgs] mapping parent/child relations without massaging the semantic labels
|
2022-05-09 16:06:04 +02:00 |
Claudio Atzori
|
2ade69dea6
|
EOSC Services - minor
|
2022-05-02 17:03:31 +02:00 |
Claudio Atzori
|
b6a7ff3a99
|
EOSC Services - removed fields from mapping, testing preparation
|
2022-05-02 15:52:33 +02:00 |
Claudio Atzori
|
f5f532d134
|
EOSC Services - ongoing update
|
2022-04-29 12:25:24 +02:00 |
Claudio Atzori
|
5ffc24d1ba
|
EOSC Services - ongoing update
|
2022-04-26 16:18:41 +02:00 |
Claudio Atzori
|
05fafa1408
|
[graph raw] avoid NPEs importing datasource consent fields
|
2022-04-06 15:23:50 +02:00 |
Claudio Atzori
|
0a0ae84c22
|
[graph raw] DOI based instance URLs on https
|
2022-03-29 10:52:58 +02:00 |
Claudio Atzori
|
f25407bbe2
|
added mapping for datasource consent fields to integrate them in the graph
|
2022-03-11 09:32:42 +01:00 |
Alessia Bardi
|
600ede1798
|
serialisation of APCs int he XML records
|
2022-02-11 11:00:20 +01:00 |
Miriam Baglioni
|
aae667e6b6
|
[APC at the result level] added the APC at the level of the result and modified test class
|
2022-02-04 12:34:25 +01:00 |
Claudio Atzori
|
f0ea2410e5
|
improved mapping titles from datacite records to consider title types
|
2022-01-21 10:50:34 +01:00 |
Sandro La Bruzzo
|
bf880e2508
|
[scala-refactor] Module dhp-graph-mapper:
Moved all scala source into src/main/scala and src/test/scala
|
2021-12-06 13:57:41 +01:00 |
Sandro La Bruzzo
|
034304b33a
|
conflict resolved on merge
|
2021-10-26 09:40:47 +02:00 |
Claudio Atzori
|
d147295c2f
|
avoiding java.io.NotSerializableException: java.util.HashMap
|
2021-10-21 14:15:57 +02:00 |
Claudio Atzori
|
3702fe478d
|
cleanup
|
2021-10-21 12:05:02 +02:00 |
Sandro La Bruzzo
|
ac36aa7d1c
|
fixed wrong Encoding during a map phase
|
2021-10-21 11:35:02 +02:00 |
Claudio Atzori
|
00b78b9c58
|
cleanup: mapping contents in the graph already defined in the OAF graph model doesn't require to be aware of the vocabularies
|
2021-10-20 14:04:45 +02:00 |
Claudio Atzori
|
c01dd0c925
|
registered oaf model classes for the KryoSerializer
|
2021-10-20 13:55:07 +02:00 |
Claudio Atzori
|
515e068a78
|
Merge branch 'beta' into hierarchical_orgs_relations
|
2021-10-19 16:46:06 +02:00 |
Claudio Atzori
|
512e7b0170
|
code formatting
|
2021-10-19 16:19:29 +02:00 |
Claudio Atzori
|
98f37c8d81
|
WIP: worflow nodes for including Scholexplorer records in the RAW graph
|
2021-10-19 16:14:40 +02:00 |
Claudio Atzori
|
7a73010acd
|
WIP: worflow nodes for including Scholexplorer records in the RAW graph
|
2021-10-19 11:59:16 +02:00 |
miconis
|
5f780a6ba1
|
bug fix in migrate entities: parameter name was wrong
|
2021-10-18 23:30:40 +02:00 |
miconis
|
995c1eddaf
|
minor change
|
2021-10-13 17:07:10 +02:00 |
miconis
|
326bf63775
|
integration of parent child orgs relations
|
2021-10-13 12:24:48 +02:00 |
Alessia Bardi
|
ccf4103a25
|
keep the original url if the decoder fails for any reason
|
2021-08-25 10:07:58 +02:00 |
Alessia Bardi
|
931f430129
|
Merge branch 'beta' into datasource_model_eosc_beta
|
2021-08-23 11:57:21 +02:00 |
Alessia Bardi
|
4c1474e693
|
Dealing with #6859#note-2: we have to decode URLs to avoid & and other chars encoded becasue of the original XML representation of data
|
2021-08-20 17:03:30 +02:00 |
Claudio Atzori
|
8cdce59e0e
|
[graph raw] let the mapping exceptions propagate
|
2021-08-12 11:32:26 +02:00 |
Claudio Atzori
|
2ee21da43b
|
suggestions from SonarLint
|
2021-08-11 12:13:22 +02:00 |
Claudio Atzori
|
19620eed46
|
applying PR#131, Patch the identifiers (source/target) in the relations, refinements
|
2021-07-30 11:09:32 +02:00 |
Claudio Atzori
|
e87e1805c4
|
[raw_all] added extra workflow step for patching the identifiers in the relations, given an id mapping dataset
|
2021-07-29 12:13:06 +02:00 |
Michele Artini
|
e6f1773d63
|
mapping of new eosc fields
|
2021-07-28 11:17:11 +02:00 |
Claudio Atzori
|
65934888a1
|
adding record identifier among the originalIds regardless of what IdentifierFactory produces
|
2021-07-19 17:52:52 +02:00 |
Claudio Atzori
|
0977baf41d
|
contents mapped from the stores with 'claim' interpretation will not change their identifier along their way towards the graph
|
2021-07-19 17:43:52 +02:00 |
Claudio Atzori
|
b7b8e0986e
|
[raw_all] The claim merge procedure includes the claimed contexts in the merged result
|
2021-07-08 10:42:31 +02:00 |
Claudio Atzori
|
fdcff42e46
|
[raw_all] Aggregator graph creation merges claims (updates) with the corresponding entity
|
2021-07-07 19:01:59 +02:00 |
Claudio Atzori
|
32bdfdccbc
|
[raw_all] Aggregator graph creation merges claims (updates) with the corresponding entity
|
2021-07-07 11:08:27 +02:00 |
Claudio Atzori
|
f580cb77e1
|
added mapping for claim relation 'resultResult_publicationDataset_isRelatedTo' (present on BETA)
|
2021-07-06 21:11:11 +02:00 |
Claudio Atzori
|
50fc5a64a0
|
[raw_all] Aggregator graph creation merges claims (updates) with the corresponding entity
|
2021-06-23 11:49:42 +02:00 |
Claudio Atzori
|
7243a40c88
|
code formatting
|
2021-06-16 15:03:03 +02:00 |
Michele Artini
|
ada063ce70
|
fixed a problem with empty mdstore list (2)
|
2021-06-14 12:04:47 +02:00 |
Michele Artini
|
83132ee99a
|
fixed a problem with empty mdstore list
|
2021-06-14 11:57:00 +02:00 |
Claudio Atzori
|
2039bb9f5f
|
orcid / orcid_pending cleaning backported from master branch
|
2021-06-14 09:40:50 +02:00 |
Michele Artini
|
ede2749822
|
orcid pid type
|
2021-06-01 12:42:43 +02:00 |
Michele Artini
|
f0fbfdcfae
|
Merge branch 'stable_ids' into import_new_mdstores
|
2021-06-01 12:03:00 +02:00 |
Michele Artini
|
03a510859a
|
removed coalesce(1)
|
2021-05-31 14:10:51 +02:00 |
Michele Artini
|
e9f2b6037c
|
patch of mdstore records
|
2021-05-31 11:36:26 +02:00 |
Michele Artini
|
ad56a44fda
|
save as gzipped sequence file
|
2021-05-28 14:45:39 +02:00 |
Michele Artini
|
4fa5671d16
|
first implementation of Hdfs Mdstores Importer
|
2021-05-27 16:22:07 +02:00 |
Claudio Atzori
|
5e4b91d9ef
|
more pervasive use of constants from ModelConstants, especially for ORCID
|
2021-05-26 18:20:23 +02:00 |
Claudio Atzori
|
9d725efdc1
|
reverted implementation of the mdstore client
|
2021-05-20 18:26:09 +02:00 |
Claudio Atzori
|
ae5c28e54f
|
code formatting
|
2021-05-20 16:13:06 +02:00 |
Claudio Atzori
|
232dce83db
|
fixes #6701: xpath for titles to support both datacite and Guidelines v4 mapping
|
2021-05-20 14:41:15 +02:00 |
Claudio Atzori
|
d4c3476152
|
mapping datasource.journal only when an issn is available, null otherwhise
|
2021-05-11 11:08:54 +02:00 |
Claudio Atzori
|
d1cbee8413
|
imported methods from CleaningFunctions, defined in GraphCleaningFunctions
|
2021-05-10 16:43:39 +02:00 |
Claudio Atzori
|
dccaf173cf
|
fixed mapping applied to ODF records. Added unit test to verify the mapping for OpenTrials
|
2021-05-05 16:36:15 +02:00 |
Claudio Atzori
|
923d19ea8e
|
mdstore read lock/unlock when bulk copying records from mongodb to hdfs
|
2021-05-04 18:06:21 +02:00 |
Claudio Atzori
|
5afa7d3e0c
|
core utilities in dhp-common moved in external module dhp-schemas
|
2021-04-27 15:44:01 +02:00 |
Claudio Atzori
|
c25238480c
|
making ODF record parsing namespace unaware (#6629)
|
2021-04-23 17:34:57 +02:00 |
miconis
|
0393cdce42
|
addition of alternative names in export queries
|
2021-04-20 12:45:21 +02:00 |
miconis
|
bf685d849f
|
addition of pids in the query for the export of openorgs for the provision, addition of ec_fields in the openorgs model
|
2021-04-07 14:27:43 +02:00 |
miconis
|
eaaefb8b4c
|
implementation of the procedure to reuse content of different dbs when creating the raw graph
|
2021-04-06 14:35:51 +02:00 |
miconis
|
c39c82dfe9
|
modification of the jobs for the integration of openorgs in the provision, dedup records are no more created by merging but simply taking results of openorgs portal
|
2021-04-06 14:31:00 +02:00 |
Claudio Atzori
|
7941d7be29
|
WIP: using common definitions from ModelConstants
|
2021-03-31 18:33:57 +02:00 |
Claudio Atzori
|
72ce741ea6
|
WIP: using common definitions from ModelConstants
|
2021-03-31 17:07:13 +02:00 |
miconis
|
2709d08fc2
|
Merge branch 'stable_ids' into openorgswf
|
2021-03-29 16:39:07 +02:00 |
miconis
|
2355cc4e9b
|
minor changes and bug fix
|
2021-03-29 10:07:12 +02:00 |
miconis
|
28c1cdd132
|
merged stable_ids into openorgswf
|
2021-03-25 10:44:49 +01:00 |
miconis
|
348b0ef921
|
bug fix, implementation of the workflow for the creation of raw_organizations (openorgs dedup), addition of the pid lists to the openorgs postgres db
|
2021-03-24 15:51:27 +01:00 |
Sandro La Bruzzo
|
c73072079d
|
fix conflicts
|
2021-03-22 16:36:31 +01:00 |
Claudio Atzori
|
8257f9a2bc
|
result.pid: adjusted the mapping applied to the contents from the aggregator
|
2021-03-17 12:45:38 +01:00 |
Claudio Atzori
|
640b885706
|
added instance.alternativeIdentifiers to the graph model, adjusted the mapping applied to the contents from the aggregator
|
2021-03-16 14:19:32 +01:00 |
Claudio Atzori
|
01630f638d
|
IdentifierFactory implementation based on the list of datasources authoritative for a given pid type
|
2021-03-09 17:11:50 +01:00 |
Claudio Atzori
|
59532b0919
|
[#6281 Provenance of product PIDs] Added PIDs to the Instance type; extended mapping for OAF/ODF records
|
2021-03-09 11:14:45 +01:00 |
Claudio Atzori
|
d525785497
|
[#6282 open access status in the Graph] Result.Instance.accessRight defined with dedicated data type that includes the open access color.
|
2021-03-09 11:12:55 +01:00 |
Claudio Atzori
|
f468c7f0d7
|
merged from master
|
2021-03-09 09:12:41 +01:00 |
Claudio Atzori
|
8d2bb24512
|
merged from master
|
2021-03-08 15:44:34 +01:00 |
miconis
|
1a85020572
|
bug fix in graph-mapper, changes in the implementation of the openorgs wf to create relations and populate openorgs db
|
2021-02-26 10:19:28 +01:00 |
Claudio Atzori
|
b830e33392
|
mdstore collector plugin
|
2021-02-25 12:30:30 +01:00 |
Claudio Atzori
|
fc3fa5e343
|
implemented mdstore collector plugin
|
2021-02-24 15:07:24 +01:00 |
miconis
|
4b2124a18e
|
implementation of the openorgs wfs, implementation of the raw_all wf to migrate openorgs db entities
|
2021-02-10 11:51:50 +01:00 |
Sandro La Bruzzo
|
686e7b507c
|
Merge branch 'hadoop_aggregator' of code-repo.d4science.org:D-Net/dnet-hadoop into aggregation_on_hadoop
|
2021-01-28 10:02:13 +01:00 |
Sandro La Bruzzo
|
98b9498b57
|
Removed old messaging system not quite used from collection and Transformation workflow
code refactor
|
2021-01-28 09:51:17 +01:00 |
Sandro La Bruzzo
|
150a617bd1
|
Merge pull request 'aggregation_on_hadoop' (#90) from sandro.labruzzo/dnet-hadoop:aggregation_on_hadoop into hadoop_aggregator
Wonderfull code... You're the Best Sandro
|
2021-01-26 16:00:47 +01:00 |
Alessia Bardi
|
505477f36f
|
format code
|
2021-01-25 18:02:49 +01:00 |
Alessia Bardi
|
ded6ed8d7d
|
no ',' author, if there are no author in ODF records
|
2021-01-25 17:57:51 +01:00 |
Sandro La Bruzzo
|
a54848a59c
|
Moved Vocabulary stuff to common module
|
2021-01-25 15:43:04 +01:00 |
Claudio Atzori
|
28460c2cd1
|
using com.fasterxml.jackson.databind.ObjectMapper instead of org.codehaus.jackson.map.ObjectMapper
|
2020-12-23 16:59:52 +01:00 |
Michele Artini
|
991e675dc6
|
validation in claim rels
|
2020-12-14 15:41:25 +01:00 |
Michele Artini
|
1bc9adc10d
|
default trust for validated rels
|
2020-12-09 16:18:37 +01:00 |
Michele Artini
|
5f21a356fd
|
reindent
|
2020-12-09 11:24:30 +01:00 |
Michele Artini
|
370a5e650b
|
validation attributes in resultProject relations
|
2020-12-09 11:18:26 +01:00 |
Claudio Atzori
|
2c407e775e
|
GenerateEntitiesApplication can be configured to hash the id value or not
|
2020-11-30 12:00:38 +01:00 |