D-Net
/
dnet-hadoop
14
5
Fork 11
Code Issues 14 Pull Requests 13 Projects 2 Releases 2 Wiki Activity
5,697 Commits 103 Branches 20 Tags 658 MiB
Commit Graph

879 Commits

Author SHA1 Message Date
Miriam Baglioni 07a1f2b31c [TransformativeAgreement] fix to remove the file downloaded from a previous run of the workflow 2024-10-28 12:23:18 +01:00
Claudio Atzori c4e8aaca1f PidCleaner used pervasively 2024-10-08 14:58:28 +02:00
Claudio Atzori 1596d70224 [bipAffiliations] fix: added publisherInputPath to the spark action parameter specifications 2024-10-02 10:02:10 +02:00
Claudio Atzori 6e0b6a886f code formatting 2024-09-30 15:13:23 +02:00
Claudio Atzori 3854fcc5e0 Merge pull request 'New Model from affRo' (#487) from affRoModelModificationOnmain into main 
Reviewed-on: #487
 2024-09-30 14:33:31 +02:00
Miriam Baglioni 371154d74f [OpenAireAffiliations] changed the code to handle mixed model in input. To be able to update some links for as many datasources as possible. So far crossref and openapc 2024-09-30 14:29:49 +02:00
Claudio Atzori 4e9f64e01a merged from the osfPreprints_plugin branch 2024-09-30 11:24:17 +02:00
Claudio Atzori 81bfe3fe32 WIP merged beta into main 2024-09-26 09:23:44 +02:00
Miriam Baglioni 0765641979 [AffRo] used the collectedfrom openaire for all the relations imported as affRo output 2024-09-25 17:23:49 +02:00
Miriam Baglioni 7cd8171268 [AffRo] refactoring 2024-09-25 17:12:51 +02:00
Miriam Baglioni a54d021c37 merge with main 2024-09-25 17:06:27 +02:00
Miriam Baglioni 6eea075324 [AffRo] changed the creation of the action set against the new model of provision of the matchings. Changed the test calss and the resources accordingly 2024-09-25 17:04:37 +02:00
Claudio Atzori 2ba67f08d3 [OpenCitations] move the extracted contents under a backup path to avoid needing to re-download it in case of errors 2024-09-25 15:39:22 +02:00
Miriam Baglioni df39360822 [AffRo] changed the creation of the action set agaisnt the nen model of provision of the matchings 2024-09-25 12:32:53 +02:00
Michele Artini fa2532db30 fixed a bug with id 2024-09-25 09:38:50 +02:00
Michele Artini b35d046fd2 fixed a bug with 'null' string 2024-09-24 15:18:54 +02:00
Michele Artini 2d7a7a962d unit test @Disabled 2024-09-23 10:19:36 +02:00
Michele Artini 6b0f7cc8b0 skip urls with authentication 2024-09-23 10:16:53 +02:00
Michele Artini 339d8124f2 osf plugin: links to contributors and primaty_file 2024-09-20 08:44:05 +02:00
Michele Artini 52bb7af03b use of dom4j 2024-09-19 14:59:05 +02:00
Michele Artini 9073b1159d partial implementation of osfPreprints plugin + tests 2024-09-19 13:58:53 +02:00
Michele Artini dcf09811a2 partial implementation of osfPreprints plugin 2024-09-19 12:42:45 +02:00
Sandro La Bruzzo 890190b7ae as described on ticket #9525 
1. Changed the mapping applied to Crossref records: anything that has a relationship "is-review-of" must be mapped as publication of type "Review".
2. Force the hostedby of Crossref records with DOI prefix 10.3410 and 10.12703 to the H1 Connect data source.
 2024-09-18 17:16:53 +02:00
Michele Artini a2fac78dcc fixed a problem in incremental harvesting 2024-09-17 10:16:28 +02:00
Michele Artini 99b7adda0c gtr2 unit test 2024-09-16 15:13:44 +02:00
Michele Artini bb9cee4f40 implementation of gtr2Publications plugin 2024-09-16 14:16:56 +02:00
Giambattista Bloisi 37b9bdc10c Fix: next returned a null value at end of stream 2024-09-05 16:52:57 +02:00
Giambattista Bloisi e7150eea7b Fix for paginationStart parameter management 2024-09-05 16:52:57 +02:00
Giambattista Bloisi 23477f3e80 Fixes for pagination strategy looping at end of download 2024-09-05 16:52:57 +02:00
Claudio Atzori ce78752aa3 BIPAffiliations to include also input data from publisher websites 2024-08-07 15:46:44 +02:00
Miriam Baglioni f1dc0050c7 [AffiliationFromPublisher]extention of test 2024-08-07 11:27:11 +02:00
Miriam Baglioni 42531afc3e [AffiliationFromPublisher]refactoring after compilation 2024-08-07 11:17:56 +02:00
Miriam Baglioni 907eeadce8 [AffiliationFromPublisher]Adding to the creation of the ActrionSet also the links got from the publishers 2024-08-07 11:08:50 +02:00
Claudio Atzori b8bc237079 [bip affiliations] considers only DOI based records 2024-08-05 12:14:06 +02:00
Claudio Atzori 8e7ef79ce0 [bip affiliations] considers only DOI based records 2024-08-05 12:13:48 +02:00
Claudio Atzori ed6d71fc70 code formatting 2024-08-05 12:12:29 +02:00
Miriam Baglioni cbe877b73c [WebCrawlAffiliation]remove from the creation of the action set the relations for pmc and pmid. Only doi are allowed 2024-08-05 11:44:38 +02:00
Claudio Atzori 97c9706469 minors 2024-08-02 15:47:56 +02:00
Claudio Atzori 07e7b9315c code formatting 2024-08-02 14:42:24 +02:00
Alessia 39810c6e7e Rest collector plugin on hadoop supports a new param to pass request headers 2024-08-02 14:41:43 +02:00
Claudio Atzori 64740475d0 depending on dhp-schemas:7.0.1 2024-07-29 11:51:42 +02:00
Miriam Baglioni 1af6571474 merging with branch beta 2024-07-25 15:48:05 +02:00
Miriam Baglioni c7f6669f1a [webcrawl] the blacklist is now in json and no more in csv after the normalization process 2024-07-25 15:20:18 +02:00
Miriam Baglioni 7cff281d3e [webcrawl] the blacklist is now in json and no more in csv after the normalization process 2024-07-25 15:16:42 +02:00
Miriam Baglioni fc60661ac5 [webcrawl] added code and test (code/resource) to verify the deletion of the relations related to results put in blacklist 2024-07-25 12:25:14 +02:00
Miriam Baglioni 6f1801d7d1 [webcrawl]- 2024-07-23 17:34:48 +02:00
Miriam Baglioni 19806c2ae3 [SDG]fixed switch of methods 2024-07-23 17:12:55 +02:00
Miriam Baglioni 9573bf576d [SDG]added code to ingest also the SDG without DOI 2024-07-23 12:47:57 +02:00
Miriam Baglioni 79985ad197 [Crossref]added mapping for DFG versus the unidentified project [https://support.openaire.eu/issues/9926?next_issue_id=9924&prev_issue_id=9927#note-4] 2024-07-17 18:30:24 +02:00
Claudio Atzori 06e3985b77 merged from beta 2024-07-17 12:01:40 +02:00