D-Net
/
dnet-hadoop
14
5
Fork 11
Code Issues 14 Pull Requests 13 Projects 2 Releases 2 Wiki Activity
5,517 Commits 111 Branches 20 Tags 660 MiB
Commit Graph

4580 Commits

Author SHA1 Message Date
Miriam Baglioni 7cd8171268 [AffRo] refactoring 2024-09-25 17:12:51 +02:00
Miriam Baglioni a54d021c37 merge with main 2024-09-25 17:06:27 +02:00
Miriam Baglioni 6eea075324 [AffRo] changed the creation of the action set against the new model of provision of the matchings. Changed the test calss and the resources accordingly 2024-09-25 17:04:37 +02:00
Claudio Atzori 2ba67f08d3 [OpenCitations] move the extracted contents under a backup path to avoid needing to re-download it in case of errors 2024-09-25 15:39:22 +02:00
Miriam Baglioni df39360822 [AffRo] changed the creation of the action set agaisnt the nen model of provision of the matchings 2024-09-25 12:32:53 +02:00
Claudio Atzori 5fdc286eb9 Merge pull request 'ticket #9525: Update Crosserf Mapping' (#479) from ticket#9525 into main 
Reviewed-on: #479
 2024-09-25 11:32:20 +02:00
Claudio Atzori e7f6eb82df Merge pull request 'fixed a bug with topic ENRICH/MORE/SUBJECT/ARXIV' (#481) from enrich_more_subject_bug into main 
Reviewed-on: #481
 2024-09-24 08:56:50 +02:00
Michele Artini 0c66b8589d removed the deletedByInference=true filter 2024-09-23 13:01:45 +02:00
Michele Artini 758d4acd05 fixed a bug with topic ENRICH/MORE/SUBJECT/ARXIV 2024-09-23 09:47:29 +02:00
Sandro La Bruzzo 890190b7ae as described on ticket #9525 
1. Changed the mapping applied to Crossref records: anything that has a relationship "is-review-of" must be mapped as publication of type "Review".
2. Force the hostedby of Crossref records with DOI prefix 10.3410 and 10.12703 to the H1 Connect data source.
 2024-09-18 17:16:53 +02:00
Claudio Atzori c648531ccb run mergeResultsOfDifferentTypes only when checkDelegatedAuthority is true 2024-09-16 16:16:23 +02:00
Giambattista Bloisi 37b9bdc10c Fix: next returned a null value at end of stream 2024-09-05 16:52:57 +02:00
Giambattista Bloisi e7150eea7b Fix for paginationStart parameter management 2024-09-05 16:52:57 +02:00
Giambattista Bloisi 23477f3e80 Fixes for pagination strategy looping at end of download 2024-09-05 16:52:57 +02:00
Claudio Atzori ce78752aa3 BIPAffiliations to include also input data from publisher websites 2024-08-07 15:46:44 +02:00
Miriam Baglioni f1dc0050c7 [AffiliationFromPublisher]extention of test 2024-08-07 11:27:11 +02:00
Miriam Baglioni 42531afc3e [AffiliationFromPublisher]refactoring after compilation 2024-08-07 11:17:56 +02:00
Miriam Baglioni 907eeadce8 [AffiliationFromPublisher]Adding to the creation of the ActrionSet also the links got from the publishers 2024-08-07 11:08:50 +02:00
Claudio Atzori b8bc237079 [bip affiliations] considers only DOI based records 2024-08-05 12:14:06 +02:00
Claudio Atzori ed6d71fc70 code formatting 2024-08-05 12:12:29 +02:00
Miriam Baglioni cbe877b73c [WebCrawlAffiliation]remove from the creation of the action set the relations for pmc and pmid. Only doi are allowed 2024-08-05 11:44:38 +02:00
Claudio Atzori 97c9706469 minors 2024-08-02 15:47:56 +02:00
Claudio Atzori 07e7b9315c code formatting 2024-08-02 14:42:24 +02:00
Alessia 39810c6e7e Rest collector plugin on hadoop supports a new param to pass request headers 2024-08-02 14:41:43 +02:00
Claudio Atzori e0f58afd30 [graph provision] include only FoS L1..L2 in the record serialization 2024-08-02 10:58:57 +02:00
Claudio Atzori 60cf7d86a1 [graph provision] include only FoS L1..L2 in the record serialization 2024-08-02 10:58:47 +02:00
Miriam Baglioni 8f11dfe554 [UnpayWall]added othe : in the identifier construction 2024-07-16 18:18:38 +02:00
Claudio Atzori d20a5e020a [graph provision] log the Solr admin application operations for alias deletion and creation 2024-07-15 16:31:04 +02:00
Claudio Atzori 3d1d8e6036 renamed workflow to better reflect its purpose 2024-07-15 15:24:18 +02:00
Claudio Atzori b70a440aca renamed class, updated criteria to consider the ORCIDs used in the matchers 2024-07-12 17:09:01 +02:00
Michele Artini 36c3df1652 tests 2024-07-12 15:29:45 +02:00
Claudio Atzori 2f13683285 [broker] fine tuned the workflow memory settings 2024-07-12 10:27:24 +02:00
Claudio Atzori 5ab409dcab [metadata collection] added -Dcom.sun.security.enableAIAcaIssuers=true as a default for metadata collection 2024-07-12 10:26:32 +02:00
Claudio Atzori 51d6a541bd [metadata collection] added the possibility to specify the JAVA_HOME and the JAVA_OPTS parameters 2024-07-11 15:24:29 +02:00
Claudio Atzori 07ce92cef2 [OAI-PMH] fixed node name 2024-07-11 11:00:23 +02:00
Miriam Baglioni f043b7b096 [Irish Tender]changed the irish.json file according to comments #26, #29, and #34 for 9635 2024-07-04 12:22:56 +02:00
Claudio Atzori ed97ba4565 Merge pull request '[prod] Openaire Affiliation Inference' (#453) from affRoFromRawStringmain into main 
Reviewed-on: #453
 2024-07-03 12:32:26 +02:00
Claudio Atzori 7b398a6d0b updated import of organization types from OpenOrgs 2024-07-03 11:11:35 +02:00
Claudio Atzori 13f6506ce5 Change the selection criteria for the pivot record of a group so that by best pid type becomes the first criteria. This will have the effect to slowly converge to records having DOI 2024-07-03 10:44:01 +02:00
Claudio Atzori 3d9ddaa23a importing organization types from OpenOrgs 2024-07-03 10:15:37 +02:00
Michele De Bonis ea1841fbd2 implementation of countryMatch and addition of workflow parameters 2024-07-01 09:14:32 +02:00
Miriam Baglioni 4dbce39237 [AffiliationInference]Extended the affiliation ingestion from OpenAIRE to include also the links derived from web crawl. Changed the provenance from BIP! to OpenAIRE 2024-06-29 18:51:06 +02:00
Miriam Baglioni 3ee8a7d18a [WebCrawl]moved to Constants web crawl name and id 2024-06-29 18:47:23 +02:00
Claudio Atzori ee7deb3f60 [graph provision] publicFormat worfklow parameter defined as optional 2024-06-28 14:52:43 +02:00
Claudio Atzori 157cc8be87 [graph provision] fixed serialization of the instancetypes 2024-06-28 14:21:12 +02:00
Claudio Atzori 023099a921 imported from beta 2024-06-26 11:40:16 +02:00
Claudio Atzori 786c217085 Using the updated Solr JSON payload model classes 2024-06-26 11:11:33 +02:00
Lampros Smyrnaios c858c02111 - Fix not using the "export HADOOP_USER_NAME" statement in "createPDFsAggregated.sh", which caused permission-issues when creating tables with Impala. 
- Remove unused "--user" parameter in "impala-shell" calls.
- Code polishing.
 2024-06-26 10:11:21 +02:00
Claudio Atzori 8220e27110 Merge pull request 'Align Solr JSON records to the explore portal requirements' (#448) from json_payload into beta_to_master_may2024 
Reviewed-on: #448
 2024-06-25 09:57:40 +02:00
Claudio Atzori 1dc7458de2 added JSON payload to the SolrInputDocument, updated unit tests 2024-06-24 14:48:09 +02:00