sabeel
/
dnet-hadoop
forked from D-Net/dnet-hadoop
1
0
Fork 0
Code Pull Requests Activity
3,290 Commits 55 Branches 18 Tags 649 MiB
Commit Graph

3290 Commits

Author SHA1 Message Date
Sandro La Bruzzo e8b3cb9147 Implemented method to download delta updates in EBI Links 2021-08-30 09:32:45 +02:00
Alessia Bardi ccf4103a25 keep the original url if the decoder fails for any reason 2021-08-25 10:07:58 +02:00
Sandro La Bruzzo 45898c71ac fixed wrong doi in pubmed 2021-08-24 15:20:04 +02:00
Alessia Bardi 00a28c0080 originalId was renamed to acronym 2021-08-23 15:02:21 +02:00
Alessia Bardi f19b04d41b code formatting after mvn compile 2021-08-23 14:33:39 +02:00
Alessia Bardi 412d2cb16a added dependencies to classgraph and opencsv. Bumped version of dhp-schemas 2021-08-23 14:32:00 +02:00
Alessia Bardi 3bcac7e88c Merge pull request 'towards EOSC datasource profiles' (#130) from datasource_model_eosc_beta into beta 
Reviewed-on: D-Net/dnet-hadoop#130
 2021-08-23 11:58:34 +02:00
Alessia Bardi 931f430129 Merge branch 'beta' into datasource_model_eosc_beta 2021-08-23 11:57:21 +02:00
Alessia Bardi 4c1474e693 Dealing with #6859#note-2: we have to decode URLs to avoid & and other chars encoded becasue of the original XML representation of data 2021-08-20 17:03:30 +02:00
Miriam Baglioni 5f8ccbc365 Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta 2021-08-20 11:13:47 +02:00
Miriam Baglioni 882abb40e4 CrossrefDump - 2021-08-20 11:12:53 +02:00
Miriam Baglioni 45c62609af CrossrefDump - modified because parameter file was moved 2021-08-20 11:12:31 +02:00
Miriam Baglioni 35880c0e7b CrossrefDump - changed the wf to be able to resume from one of the steps 2021-08-20 11:11:35 +02:00
Miriam Baglioni f3b6c392c1 CrossrefDump - moving parameter file under folder crossref_dump_reader 2021-08-20 11:10:58 +02:00
Miriam Baglioni 65822400ce CrossrefDump - added new parameter file that was missing 2021-08-20 11:10:35 +02:00
Alessia Bardi a053e1513c different funders in blacklist from BETA and PROD aggregator 2021-08-19 11:32:27 +02:00
Alessia Bardi 812bd54c57 different funders in blacklist from BETA and PROD aggregator 2021-08-19 11:30:14 +02:00
Miriam Baglioni a65d3caaea Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta 2021-08-19 10:29:10 +02:00
Miriam Baglioni e5cf11d088 change open access route to result matching hbm to gold 2021-08-19 10:29:04 +02:00
Claudio Atzori 7c0c67bdd6 added mock pom 2021-08-13 17:45:53 +02:00
Claudio Atzori 82086f3422 fixed directory name 2021-08-13 17:42:14 +02:00
Claudio Atzori bc7068106c added crossref download oozie workflow 2021-08-13 17:19:44 +02:00
Claudio Atzori 2c0a05f11a manually merged PR#139 2021-08-13 17:15:53 +02:00
Claudio Atzori d43667d857 Merge pull request 'Automatic download of Crossref' (#138) from crossref_dw_wf into beta 
Reviewed-on: D-Net/dnet-hadoop#138
 2021-08-13 17:10:10 +02:00
Miriam Baglioni 5856ca8a7b merging with branch beta - resolved conflicts 2021-08-13 16:45:45 +02:00
Miriam Baglioni 6fec71e8d2 removed the specific of the infra we are running the wf from the wf name 2021-08-13 16:39:02 +02:00
Miriam Baglioni ed7e28490a change in sh 2021-08-13 16:19:01 +02:00
Claudio Atzori 7743d0f919 consolidated dnet wf profiles into the same submodule 2021-08-13 16:14:54 +02:00
Miriam Baglioni 6eb7508995 mergin with branch beta 2021-08-13 16:07:04 +02:00
Claudio Atzori f74adc4752 added DownloadCSV2 as alternative implementation of the same download procedure 2021-08-13 15:52:15 +02:00
Claudio Atzori 5f0903d50d fixed CSV downloader & tests 2021-08-13 14:17:54 +02:00
Claudio Atzori 17cefe6a97 [HBM] removed stale replace option 2021-08-13 12:43:59 +02:00
Claudio Atzori 7ee2757fcd fixed DownloadCSV parameters spec; workflow patching the hostedby replaces the graph content (publication, datasource) rather than creating a copy 2021-08-13 12:41:01 +02:00
Claudio Atzori c3ad4ab701 minor fixes 2021-08-13 12:23:15 +02:00
Claudio Atzori baed5e3337 test classes moved in specific components 2021-08-13 12:14:47 +02:00
Claudio Atzori 3359f73fcf cleanup & best practices 2021-08-13 12:00:42 +02:00
Claudio Atzori 4e6575a428 Merge pull request 'Moving Download CSV' (#137) from refactoring_download_csv into beta 
Reviewed-on: D-Net/dnet-hadoop#137
 2021-08-13 10:41:01 +02:00
Miriam Baglioni f4ec81c92c mergin with branch beta 2021-08-13 10:31:35 +02:00
Miriam Baglioni dc8b05b39e Hosted By Map - changed the association with the datasource id for the hostedby element: there is no more the need to compute it. With the new HBM it is already the id in the graph 2021-08-13 10:18:25 +02:00
Miriam Baglioni 32fd75691f refactoring 2021-08-13 10:15:42 +02:00
Miriam Baglioni dfd1e53c69 added external dependency for version 2021-08-13 10:15:12 +02:00
Miriam Baglioni 01db1f8bc4 GetCSV refactoring - removed not needed import 2021-08-13 10:14:17 +02:00
Miriam Baglioni 964a46ca21 GetCSV refactoring - modified due to movement of classes 2021-08-13 10:11:18 +02:00
Miriam Baglioni eaf077fc34 GetCSV refactoring - removed not needed dependency 2021-08-13 10:08:58 +02:00
Miriam Baglioni 5f674efb0c moved dependency version in external pom 2021-08-13 10:07:53 +02:00
Miriam Baglioni 5cd5714530 GetCSV refactoring - added ignore annotation for fields not in input csv 2021-08-13 10:06:49 +02:00
Miriam Baglioni 58f241f4a2 GetCSV refactoring - changed due to change of input resource 2021-08-13 10:04:44 +02:00
Miriam Baglioni f3d575f749 GetCSV refactoring - changed due to changes in input resource 2021-08-13 10:03:57 +02:00
Miriam Baglioni a5f6edfa6c GetCSV refactoring - changed to mirror the original model class 2021-08-13 09:30:03 +02:00
Miriam Baglioni ed183d878e GetCSV refactoring - modified test classes due to change in the model of projects and programme 2021-08-13 09:28:51 +02:00