D-Net
/
dnet-hadoop
14
5
Fork 11
Code Issues 14 Pull Requests 13 Projects 2 Releases 2 Wiki Activity
3,738 Commits 111 Branches 20 Tags 660 MiB
Commit Graph

3738 Commits

Author SHA1 Message Date
Claudio Atzori bd59b58efb test for the tolerant deserialisation utility method 2022-01-04 11:26:56 +01:00
Claudio Atzori a6977197b3 serialise records in the OAF-store-graph mdstores in json format. Read them again in the graph construction phase using a tolerant parser to support backward compatible changes in the evolution of the schema 2022-01-03 17:25:26 +01:00
Miriam Baglioni 4c60ee1718 mergin with branch beta 2022-01-03 15:24:02 +01:00
Miriam Baglioni 92fd69e25d [SDG-FOS] alternative way to get input data to avoid OOM error while getting csv 2022-01-03 15:23:06 +01:00
Claudio Atzori fe7e5f4748 Merge pull request '[stats wf] result_result relations, usage stats, monitor views, indicator for sprint 5' (#179) from antonis.lempesis/dnet-hadoop:beta into beta 
Reviewed-on: #179
 2022-01-03 14:52:11 +01:00
Claudio Atzori bcea4e3a9b added dnet workflow profile for the orchestration of the simplified and complete graph construction and processing pipeline, where the IIS works on the non-deduplicated graph 2022-01-03 14:33:00 +01:00
Miriam Baglioni a706ba0c08 Merge pull request 'SDG Integration' (#178) from SDG into beta 
Reviewed-on: #178
 2021-12-23 14:50:00 +01:00
Antonis Lempesis 81ee654271 added result_result relations 2021-12-23 15:46:17 +02:00
Antonis Lempesis 7551e52e95 fixed a typo 2021-12-23 15:33:53 +02:00
Miriam Baglioni 7a1b440413 [SDG] logic to create unresolved entities out of SDG input. This changes also some classes related to FOS to reuse the same code. The code under createunresolvedentities create results with the merged update of the the inputs provided (bip at the level of the isntance, fos and sdg for subjects) 2021-12-23 13:24:28 +01:00
Claudio Atzori 278cf08421 Merge pull request 'Normalising DOI urls' (#177) from instance_group_by_url into beta 
Reviewed-on: #177
 2021-12-23 12:40:17 +01:00
Claudio Atzori cccb16900c https://support.openaire.eu/issues/7330 normalising DOI urls 2021-12-23 12:33:53 +01:00
Miriam Baglioni 2a67ee13ec [SDG] added model class 2021-12-23 10:37:52 +01:00
Miriam Baglioni 5c4fee3533 Merge pull request '[Graph Dump] fixed issue on extraction of relation between entities and contexts: the relationship name and type were swapped' (#176) from dump into beta 
Reviewed-on: #176
 2021-12-23 10:16:20 +01:00
Miriam Baglioni 69e9ea9eeb [Graph Dump] Test for extraction of rels from entities extended 2021-12-23 10:15:30 +01:00
Miriam Baglioni 31b26d48ac [Graph Dump] fixed issue on extraction of relation between entities and contexts: the relationship name and type were swapped 2021-12-23 10:09:47 +01:00
Miriam Baglioni bf3a9505e0 Merge pull request 'FOS' (#175) from FOS into beta 
Reviewed-on: #175
 2021-12-23 09:06:56 +01:00
Miriam Baglioni 10579c0dd0 [FOS]fixed doi value in test 2021-12-22 23:10:16 +01:00
Miriam Baglioni 6116fc5d40 [FOS]added logic to include only different subjects. Test refactoring and extention 2021-12-22 23:04:22 +01:00
Miriam Baglioni b81efb6a9d [FOS]changed the mapping between the csv and the model. Changed Test classes and resources 2021-12-22 21:40:35 +01:00
Miriam Baglioni 73175ba086 mergin with branch beta 2021-12-22 16:45:15 +01:00
Miriam Baglioni de6c4c8968 [FOS]creation of the unresolved entities: remove the split for the doi: no more needed since each row is related to one doi 2021-12-22 16:44:44 +01:00
Miriam Baglioni b352fbe453 Merge pull request 'bipFinder: unresolved entities' (#174) from bipFinder into beta 
Reviewed-on: #174
 2021-12-22 16:42:30 +01:00
Miriam Baglioni 34ac56565d refactoring 2021-12-22 16:28:11 +01:00
Miriam Baglioni 20ef1d657f refactoring 2021-12-22 16:26:36 +01:00
Miriam Baglioni 813f856d3f [BipFinder] removing left over parameter in wf 2021-12-22 16:11:12 +01:00
Miriam Baglioni 2c126ed014 [BipFinder] create unresolved entities with measures at the level of the instance 2021-12-22 16:03:41 +01:00
Miriam Baglioni bf52a1847b Merge pull request 'bipFinder at the level of the result' (#173) from bipFinder into beta 
Reviewed-on: #173
 2021-12-22 15:48:03 +01:00
Miriam Baglioni 0807fdb65a [BipFinder] remove not needed resources 2021-12-22 15:37:00 +01:00
Miriam Baglioni b5e11a3a0a [BipFinder] put in common package BipFinder model 2021-12-22 15:33:05 +01:00
Miriam Baglioni c5739c4266 [BipFinder] create action set for the measures at the level of the result 2021-12-22 15:08:33 +01:00
Miriam Baglioni da5f6260aa mergin with branch beta 2021-12-22 13:12:02 +01:00
Miriam Baglioni 4849270c55 Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta 2021-12-22 13:07:37 +01:00
Claudio Atzori 8d18500069 using dhp-schema:2.9.24 2021-12-22 12:47:21 +01:00
Miriam Baglioni 9d19b057b8 Merge pull request '[GRAPH DUMP]Moving Measures' (#159) from dump into beta 
Reviewed-on: #159
 2021-12-22 12:40:35 +01:00
Miriam Baglioni be0acccf42 Merge branch 'beta' into dump 2021-12-22 12:39:57 +01:00
Miriam Baglioni 89ea9fa0e1 Merge branch 'dump' of https://code-repo.d4science.org/D-Net/dnet-hadoop into dump 2021-12-22 12:36:32 +01:00
Antonis Lempesis 16539d7360 added usage stats 2021-12-22 02:54:42 +02:00
Antonis Lempesis 3edd661608 fixed column names 2021-12-21 22:55:04 +02:00
Antonis Lempesis a4c0cbb98c fixed typos in indicators. Added extra views in monitor 2021-12-21 15:54:38 +02:00
Miriam Baglioni e24a7f3496 mergin with branch beta 2021-12-21 13:57:19 +01:00
Miriam Baglioni d1ae219cb4 Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta 2021-12-21 13:55:53 +01:00
Miriam Baglioni 460e6b95d6 [Graph Dump] - 2021-12-21 13:48:03 +01:00
Sandro La Bruzzo 3920d68992 Fixed workflow generation of delta in datacite 2021-12-21 11:41:49 +01:00
Antonis Lempesis 58996972d9 added first indicator of sprint 5 2021-12-21 03:35:04 +02:00
dimitrispie c1cdec09a9 Sprint 5 and other changes 2021-12-20 19:23:57 +02:00
Miriam Baglioni 3cc1b7b153 mergin with branch beta 2021-12-15 17:25:02 +01:00
Miriam Baglioni 5e5dfd619c Merge branch 'beta' into dump 2021-12-15 17:21:55 +01:00
Miriam Baglioni 63b648b0dd Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta 2021-12-15 12:41:15 +01:00
Antonis Lempesis f0b523cfa7 removed the too restrctive clause. will discuss again 2021-12-15 12:32:15 +01:00