 Claudio Atzori
|
391aa1373b
|
added unit test
|
2022-01-19 17:13:21 +01:00 |
|
Claudio Atzori
|
44a937f4ed
|
factored out entity grouping implementation, extended to consider results from delegated authorities rather than identical records from other sources
|
2022-01-19 12:24:52 +01:00 |
 Miriam Baglioni
|
a7c4d0d16d
|
[DoiBoost Organizations] added parameter to specify the action in the wf raw_organizations to be able to load the openorgs organization as in the loading step for the construction of the graph
|
2022-01-13 13:52:00 +01:00 |
 Sandro La Bruzzo
|
57e2c4b749
|
formatted code
|
2022-01-12 09:40:28 +01:00 |
|
Claudio Atzori
|
4f212652ca
|
scalafmt: code formatting
|
2022-01-11 16:57:48 +01:00 |
|
Claudio Atzori
|
908294d86e
|
OAF-store-graph mdstores: firther fix for PR#180
|
2022-01-05 15:49:05 +01:00 |
|
Claudio Atzori
|
bd59b58efb
|
test for the tolerant deserialisation utility method
|
2022-01-04 11:26:56 +01:00 |
|
Claudio Atzori
|
a6977197b3
|
serialise records in the OAF-store-graph mdstores in json format. Read them again in the graph construction phase using a tolerant parser to support backward compatible changes in the evolution of the schema
|
2022-01-03 17:25:26 +01:00 |
|
Miriam Baglioni
|
7a1b440413
|
[SDG] logic to create unresolved entities out of SDG input. This changes also some classes related to FOS to reuse the same code. The code under createunresolvedentities create results with the merged update of the the inputs provided (bip at the level of the isntance, fos and sdg for subjects)
|
2021-12-23 13:24:28 +01:00 |
|
Miriam Baglioni
|
69e9ea9eeb
|
[Graph Dump] Test for extraction of rels from entities extended
|
2021-12-23 10:15:30 +01:00 |
|
Miriam Baglioni
|
31b26d48ac
|
[Graph Dump] fixed issue on extraction of relation between entities and contexts: the relationship name and type were swapped
|
2021-12-23 10:09:47 +01:00 |
|
Miriam Baglioni
|
be0acccf42
|
Merge branch 'beta' into dump
|
2021-12-22 12:39:57 +01:00 |
|
Miriam Baglioni
|
460e6b95d6
|
[Graph Dump] -
|
2021-12-21 13:48:03 +01:00 |
|
Sandro La Bruzzo
|
3920d68992
|
Fixed workflow generation of delta in datacite
|
2021-12-21 11:41:49 +01:00 |
|
Sandro La Bruzzo
|
b881ee5ef8
|
[scholexplorer]
- implemented generation of scholix of delta update of datacite
|
2021-12-15 11:25:32 +01:00 |
|
Sandro La Bruzzo
|
63952018c0
|
[scholexplorer]
-moved SparkRetrieveDataciteDelta in scala folder
|
2021-12-15 11:25:32 +01:00 |
|
Sandro La Bruzzo
|
e5bff64f2e
|
[scholexplorer]
- Minor fix on SparkConvertRDDtoDataset
-first implementation of retrieve datacite dump
|
2021-12-15 11:25:32 +01:00 |
|
Miriam Baglioni
|
56409d1281
|
[Dump] resolved conflicts with beta and merging
|
2021-12-14 15:03:45 +01:00 |
|
Miriam Baglioni
|
8d755cca80
|
-
|
2021-12-13 15:01:40 +01:00 |
|
Claudio Atzori
|
41c70c607d
|
cleaning workflow assigns the proper default instance type when a value could not be cleaned using the vocabularies
|
2021-12-09 16:44:28 +01:00 |
|
Claudio Atzori
|
e6e177dda0
|
vocabulary based cleaning considers also the term label when looking up for a synonym
|
2021-12-09 13:57:53 +01:00 |
|
Miriam Baglioni
|
d1df01ff1e
|
[Graph Dump] fixed resource for test
|
2021-12-06 15:15:48 +01:00 |
|
Sandro La Bruzzo
|
ed0c352799
|
[test-fixing] fixed wrong test
|
2021-12-06 15:07:41 +01:00 |
|
Sandro La Bruzzo
|
bf880e2508
|
[scala-refactor] Module dhp-graph-mapper:
Moved all scala source into src/main/scala and src/test/scala
|
2021-12-06 13:57:41 +01:00 |
|
Miriam Baglioni
|
4bb1d43afc
|
-
|
2021-12-03 12:35:51 +01:00 |
|
Claudio Atzori
|
863a2f9db3
|
avoid to filter OAF records defined as invisible = true
|
2021-12-03 09:08:12 +01:00 |
|
Miriam Baglioni
|
d9f80488cc
|
[GRAPH DUMP] Add one more test to check the filtering of the relations
|
2021-12-02 14:15:19 +01:00 |
|
Miriam Baglioni
|
58bc3f223a
|
[GRAPH DUMP] Add filtering for relation we do not want to dump. It is based on the relclass
|
2021-12-02 14:09:46 +01:00 |
|
Miriam Baglioni
|
8905a39bf3
|
mergin with branch beta
|
2021-12-02 13:17:29 +01:00 |
|
Claudio Atzori
|
3b19821f3c
|
added stats computation on the graph hive DB tables
|
2021-12-02 10:44:10 +01:00 |
|
Claudio Atzori
|
cfa4560769
|
minor: fixed hive action name
|
2021-12-02 10:43:36 +01:00 |
|
Claudio Atzori
|
d85af6fc25
|
[cleaning wf] fixed OAF record navigation, a mapping defined on a container object would have prevented the natvigation to continue on its properties
|
2021-12-01 15:49:15 +01:00 |
|
Claudio Atzori
|
014e872ae1
|
[resolution wf] added optional parameter to skip the entity resolution
|
2021-11-26 15:38:56 +01:00 |
|
Claudio Atzori
|
5c6d328537
|
code formatting
|
2021-11-26 15:38:16 +01:00 |
|
Sandro La Bruzzo
|
483d3039d1
|
entity resolution: added distcpt of missing entities in graph materialization
|
2021-11-22 15:55:24 +01:00 |
|
Sandro La Bruzzo
|
93fe8ce8b2
|
entity resolution: fix test
|
2021-11-22 15:50:43 +01:00 |
|
Sandro La Bruzzo
|
35e20b0647
|
updated resolution wf:
- generate a new version of the graph
- changed merge from union to join
|
2021-11-22 11:48:55 +01:00 |
|
Miriam Baglioni
|
fdb75b180e
|
[Cleaning] added couple of tests for DOIBOOST publications
|
2021-11-21 16:35:22 +01:00 |
|
Miriam Baglioni
|
0506fa2654
|
[Graph Dump] changed to mirror the changes in the model
|
2021-11-19 15:56:25 +01:00 |
|
Miriam Baglioni
|
9fae872181
|
[Graph Dump] changed to mirror the changes in the model
|
2021-11-19 11:25:50 +01:00 |
|
Claudio Atzori
|
bb5dca7979
|
cleanup
|
2021-11-18 17:10:46 +01:00 |
|
Miriam Baglioni
|
793b5a8e5f
|
Aggiornare 'dhp-workflows/dhp-graph-mapper/src/main/java/eu/dnetlib/dhp/oa/graph/dump/ResultMapper.java'
Removing the dump of Measure at the level of the result. We decided not to map it
|
2021-11-18 14:49:38 +01:00 |
|
Miriam Baglioni
|
5dc5792722
|
[Graph Dump] Change test resource to mirror the movement of the measure element
|
2021-11-18 14:39:12 +01:00 |
|
Miriam Baglioni
|
0136a8c266
|
[Graph Dump] Change test to mirror that measure is at the level of the isntance
|
2021-11-18 14:38:33 +01:00 |
|
Miriam Baglioni
|
1b79c0ee79
|
mergin with branch beta
|
2021-11-18 11:01:00 +01:00 |
|
Claudio Atzori
|
e0395719d7
|
Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta
|
2021-11-17 14:17:27 +01:00 |
|
Claudio Atzori
|
82a4e4efae
|
[cleaning wf] fixed methodology to rule out invalid result titles, based on https://support.openaire.eu/issues/7206
|
2021-11-17 14:17:22 +01:00 |
|
Miriam Baglioni
|
6d4a1c57ee
|
[Resolve Entities] Change test dataset to mirror the modification in the creation of the map between the pids and the unresolved
|
2021-11-17 12:41:52 +01:00 |
|
Miriam Baglioni
|
c6a9f0a1a8
|
mergin with branch beta
|
2021-11-16 12:04:40 +01:00 |
|
Miriam Baglioni
|
99d86134f5
|
[Graph Dump] changed the dump since the measures have been moded at the level of the instance
|
2021-11-16 12:04:21 +01:00 |