andreas.czerniak
/
BrBETA_dnet-hadoop
forked from D-Net/dnet-hadoop
1
1
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
3,543 Commits 23 Branches 0 Tags 90 MiB
Commit Graph

3543 Commits

Author SHA1 Message Date
Andreas Czerniak 5db5eb2c25 Adapting dc_cleaning_OPENAIREplus_compliant_hal to Hadoop 2022-08-04 16:37:40 +02:00
Andreas Czerniak 1622b40842 Merge branch 'beta' of https://code-repo.d4science.org/andreas.czerniak/BrBETA_dnet-hadoop into beta 2022-01-26 10:45:27 +01:00
Andreas Czerniak 854da6c0e7 adding xsl+records 2022-01-26 10:22:23 +01:00
Andreas Czerniak 2d0a4d5b85 refine of rateLimit, add fileGZip,dris2db.xsl 2022-01-26 10:22:23 +01:00
Andreas Czerniak c777b50995 refine of rateLimit, add fileGZip,dris2db.xsl 2022-01-25 17:00:05 +01:00
Claudio Atzori 97ad94d7d9 [graph resolution] drop output path at the beginning 2022-01-24 18:02:07 +01:00
Sandro La Bruzzo 7a3819144d Merge pull request 'title types from datacite records' (#188) from datacite_title_mapping into beta 
Reviewed-on: D-Net/dnet-hadoop#188
 2022-01-21 11:05:25 +01:00
Claudio Atzori f0ea2410e5 improved mapping titles from datacite records to consider title types 2022-01-21 10:50:34 +01:00
Claudio Atzori b37bc277c4 reintroduced the hostedby patching to the datacite records 2022-01-21 09:15:13 +01:00
Miriam Baglioni 42e8f76778 [GraphCleaning] change the return value in the filtering function to avoid to lose the APC entities 2022-01-13 16:06:43 +01:00
Miriam Baglioni a7c4d0d16d [DoiBoost Organizations] added parameter to specify the action in the wf raw_organizations to be able to load the openorgs organization as in the loading step for the construction of the graph 2022-01-13 13:52:00 +01:00
Miriam Baglioni 7bf12ad24a Merge pull request 'BipInstance' (#185) from BipInstance into beta 
Reviewed-on: D-Net/dnet-hadoop#185
 2022-01-12 18:15:38 +01:00
Miriam Baglioni a75fb8c47a [BipFinderInstanceLevel] change pom to align to the dhp-schema release 2.10.24 and refactoring 2022-01-12 18:06:26 +01:00
Miriam Baglioni 4d517ed9ec mergin with branch beta 2022-01-12 17:29:37 +01:00
Miriam Baglioni e7d5a39c03 [BipFinderInstanceLevel] added tests in test class 2022-01-12 17:25:04 +01:00
Claudio Atzori dbd6fa1d65 scalafmt: remote referencing the common definition files makes it work compiling the entire project as well as the individual submodules 2022-01-12 17:19:38 +01:00
Miriam Baglioni 4993666d73 [BipFinderInstanceLevel] changed creation of the instance to allow to enrich existing instances with same pid 2022-01-12 16:53:47 +01:00
Sandro La Bruzzo 1b9e8378b3 Merge pull request 'scalafmt: code style for scala' (#184) from scalafmt into beta 
Reviewed-on: D-Net/dnet-hadoop#184
 2022-01-12 09:58:39 +01:00
Sandro La Bruzzo 57e2c4b749 formatted code 2022-01-12 09:40:28 +01:00
Sandro La Bruzzo b78d2b71f0 updated scala format configuration 2022-01-12 09:38:34 +01:00
Claudio Atzori 0f2144b5e0 scalafmt: code formatting 2022-01-11 17:03:44 +01:00
Claudio Atzori dcd282977c pulled from beta 2022-01-11 16:59:41 +01:00
Claudio Atzori 4f212652ca scalafmt: code formatting 2022-01-11 16:57:48 +01:00
Sandro La Bruzzo 0163dadb7f [doiboost] 
- update MAG schema, new filed added on version dec-2021
 2022-01-11 11:05:44 +01:00
Miriam Baglioni 904e1c2667 Merge pull request 'Affiliation Propagation through semantic relation' (#183) from enrichment into beta 
Reviewed-on: D-Net/dnet-hadoop#183
 2022-01-07 19:18:16 +01:00
Miriam Baglioni 064f9bbd87 [AFFPropSR] added new paprameter for the number of iterations and new code for just one iteration 2022-01-07 18:58:51 +01:00
Miriam Baglioni 93f26fb742 Merge pull request '[SDG-FOS] to import SDG file not considering the header' (#182) from SDG into beta 
Reviewed-on: D-Net/dnet-hadoop#182
 2022-01-07 16:28:55 +01:00
Miriam Baglioni b7e450070b [SDG-FOS] to import SDG file not considering the header 2022-01-07 12:13:26 +01:00
Miriam Baglioni af8a33638d Merge pull request 'SDG - FOS' (#181) from SDG into beta 
Reviewed-on: D-Net/dnet-hadoop#181
 2022-01-07 11:31:19 +01:00
Miriam Baglioni 639190370a mergin with branch beta 2022-01-07 11:29:25 +01:00
Miriam Baglioni adccc2346a [SDG-FOS] to lower case for the doi 2022-01-07 11:28:50 +01:00
Claudio Atzori 908294d86e OAF-store-graph mdstores: firther fix for PR#180 2022-01-05 15:49:05 +01:00
Claudio Atzori 58f8998e3d OAF-store-graph mdstores: save them in text format 2022-01-04 15:02:09 +01:00
Claudio Atzori 174c3037e1 OAF-store-graph mdstores: save them in text format 2022-01-04 14:40:16 +01:00
Claudio Atzori 045d767013 OAF-store-graph mdstores: save them in text format 2022-01-04 14:23:01 +01:00
Claudio Atzori cb30770a0b Merge pull request 'tolerant parsing of OAF-store-graph mdstores' (#180) from graph_interpretation_mdstores into beta 
Reviewed-on: D-Net/dnet-hadoop#180
 2022-01-04 11:32:29 +01:00
Claudio Atzori bd59b58efb test for the tolerant deserialisation utility method 2022-01-04 11:26:56 +01:00
Claudio Atzori a6977197b3 serialise records in the OAF-store-graph mdstores in json format. Read them again in the graph construction phase using a tolerant parser to support backward compatible changes in the evolution of the schema 2022-01-03 17:25:26 +01:00
Miriam Baglioni 4c60ee1718 mergin with branch beta 2022-01-03 15:24:02 +01:00
Miriam Baglioni 92fd69e25d [SDG-FOS] alternative way to get input data to avoid OOM error while getting csv 2022-01-03 15:23:06 +01:00
Claudio Atzori fe7e5f4748 Merge pull request '[stats wf] result_result relations, usage stats, monitor views, indicator for sprint 5' (#179) from antonis.lempesis/dnet-hadoop:beta into beta 
Reviewed-on: D-Net/dnet-hadoop#179
 2022-01-03 14:52:11 +01:00
Claudio Atzori bcea4e3a9b added dnet workflow profile for the orchestration of the simplified and complete graph construction and processing pipeline, where the IIS works on the non-deduplicated graph 2022-01-03 14:33:00 +01:00
Miriam Baglioni a706ba0c08 Merge pull request 'SDG Integration' (#178) from SDG into beta 
Reviewed-on: D-Net/dnet-hadoop#178
 2021-12-23 14:50:00 +01:00
Antonis Lempesis 81ee654271 added result_result relations 2021-12-23 15:46:17 +02:00
Antonis Lempesis 7551e52e95 fixed a typo 2021-12-23 15:33:53 +02:00
Miriam Baglioni 7a1b440413 [SDG] logic to create unresolved entities out of SDG input. This changes also some classes related to FOS to reuse the same code. The code under createunresolvedentities create results with the merged update of the the inputs provided (bip at the level of the isntance, fos and sdg for subjects) 2021-12-23 13:24:28 +01:00
Claudio Atzori 278cf08421 Merge pull request 'Normalising DOI urls' (#177) from instance_group_by_url into beta 
Reviewed-on: D-Net/dnet-hadoop#177
 2021-12-23 12:40:17 +01:00
Claudio Atzori cccb16900c https://support.openaire.eu/issues/7330 normalising DOI urls 2021-12-23 12:33:53 +01:00
Miriam Baglioni 2a67ee13ec [SDG] added model class 2021-12-23 10:37:52 +01:00
Miriam Baglioni 5c4fee3533 Merge pull request '[Graph Dump] fixed issue on extraction of relation between entities and contexts: the relationship name and type were swapped' (#176) from dump into beta 
Reviewed-on: D-Net/dnet-hadoop#176
 2021-12-23 10:16:20 +01:00