lsmyrnaios
/
dnet-hadoop
forked from D-Net/dnet-hadoop
1
0
Fork 0
Code Pull Requests Activity
5,480 Commits 68 Branches 18 Tags 877 MiB
Commit Graph

5480 Commits

Author SHA1 Message Date
sandro.labruzzo ac8995ab64 Merge remote-tracking branch 'origin/beta' into crossref_mapping_improvement 2024-11-20 09:52:51 +01:00
sandro.labruzzo 496007188a Added assertion on CrossrefMappingTest 2024-11-20 09:50:09 +01:00
sandro.labruzzo a1297082e2 Crossref Enhancements: 
-Accurate Review Type Assignment: Resolved an issue identified in ticket https://support.openaire.eu/issues/9525#note-13. When a relationship of "is-review-of" is detected, the publication type is now correctly set to "Review."
-Enhanced Author Affiliation Data: Implemented Miriam's suggestion by including a new field, "RawAffiliationString," in each author entry. This additional data provides a more granular level of detail regarding author affiliations, potentially improving discoverability and research analysis.
 2024-11-19 14:57:18 +01:00
Claudio Atzori cf7d9a32ab disable autoBroadcastJoin in the cleaning workflow 2024-11-15 09:17:28 +01:00
Claudio Atzori 5f512f510e code formatting 2024-11-15 09:16:51 +01:00
Claudio Atzori b95672b420 mergeUtils set the result identifier when enforcing the result type 2024-11-15 09:16:18 +01:00
Claudio Atzori 9e8849b753 Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta 2024-11-13 20:41:51 +01:00
Claudio Atzori 4a3b173ca2 defaults to 0000 - Unknown in case the instance type lookup in the dnet:result_typologies doesn't find a corresponding result type binding 2024-11-13 16:27:00 +01:00
Giambattista Bloisi 5ee8881646 Merge pull request '[danishfunders] added link for danish funders versus the unidentified project for IRFD (501100004836) CF (501100002808) and NNF(501100009708)' (#502) from danishFunders_crossrefmap into beta 
Reviewed-on: D-Net/dnet-hadoop#502
 2024-11-13 12:01:38 +01:00
Miriam Baglioni fb1f0f8850 [danishfunders] added the possibility to link also versus a specif award if present in the metadata 2024-11-13 12:00:33 +01:00
Giambattista Bloisi 5b4d821bf9 Merge pull request 'Crossref: generate canonical openaire id for results in affiliation relationship' (#507) from fix_crossref_affiliations into beta 
Reviewed-on: D-Net/dnet-hadoop#507
 2024-11-13 11:01:37 +01:00
Giambattista Bloisi 03c262ccb9 Crossref: generate canonical openaire id for results in affiliation relationship 2024-11-13 10:56:17 +01:00
Claudio Atzori 07f267bb10 fix vocabulary lookup in mergeutils 2024-11-13 08:14:26 +01:00
Claudio Atzori 8088943399 Merge pull request 'enforce resulttype' (#506) from merge_resulttypes into beta 
Reviewed-on: D-Net/dnet-hadoop#506
 2024-11-12 14:20:22 +01:00
Claudio Atzori 6c5df761e2 enforce resulttype based on the dnet:result_typologies vocabulary and upon merge 2024-11-12 14:18:04 +01:00
Claudio Atzori 9f7a606ddd Merge pull request 'betaFixPerson' (#505) from betaFixPerson into beta 
Reviewed-on: D-Net/dnet-hadoop#505
 2024-11-12 14:09:22 +01:00
Miriam Baglioni 250f101779 [person] fixed issue in creating project identifier for the graph for person->project relations 2024-11-11 16:04:06 +01:00
Miriam Baglioni f1ea9da5bc [person] checked type in inferenceprovenance 2024-11-11 15:37:56 +01:00
Miriam Baglioni b0283fe94c [person] fix provenance of pid in person when it is orcid (classid entityregistry to avoid the cleaning put orcid_pending) 2024-11-11 14:57:57 +01:00
Giambattista Bloisi f31f22801f Merge pull request 'Remove ORCID information when the same ORCID ID is used multiple times in the same result for different authors' (#503) from clean_clashing_orcids into beta 
Reviewed-on: D-Net/dnet-hadoop#503
 2024-11-08 09:31:11 +01:00
Miriam Baglioni 6fd9ec8566 [danishfunders] added link for danish funders versus the unidentified project for IRFD (501100004836) CF (501100002808) and NNF(501100009708) 2024-11-07 13:55:31 +01:00
Giambattista Bloisi 8f5171557e Remove ORCID information when the same ORCID ID is used multiple times in the same result for different authors 2024-11-07 12:22:34 +01:00
Claudio Atzori f7bb53fe78 [orcid enrichment] added missing workflow parameter: workingDir 2024-11-07 01:04:43 +01:00
Claudio Atzori 973aa7dca6 [dedup] force the Relation schema when reading the merge rels 2024-11-06 12:29:06 +01:00
Claudio Atzori a42c8b7c85 person table directory produced by the workflows raw_all and merge graphs 2024-10-30 11:25:17 +01:00
Claudio Atzori a877c76d70 make MergeUtils.selectOldestDate less prone to errors when receiving invalid date formats 2024-10-30 11:24:25 +01:00
Claudio Atzori 26cdc7e439 Avoid NPEs in MergeUtils 2024-10-30 07:35:47 +01:00
Claudio Atzori 323c76eafc patch relations job: removed non necessary logging 2024-10-30 07:35:30 +01:00
Miriam Baglioni 69aee609ef [bulktag] align type to community api 2024-10-29 15:53:04 +01:00
Claudio Atzori 5ca031c8d6 [graph raw] rule out empty PIDs 2024-10-29 13:48:41 +01:00
Claudio Atzori 499892b67c [graph raw] rule out empty PIDs 2024-10-29 09:51:30 +01:00
Claudio Atzori e4504fd98d [Person] fixed project identifier creation 2024-10-28 15:32:09 +01:00
Claudio Atzori 9b4415cb67 using _the right_ scala 2.11 converters 2024-10-28 13:56:25 +01:00
Claudio Atzori e6ca382deb using scala 2.11 converters 2024-10-28 13:52:06 +01:00
Claudio Atzori 940735921f Merge pull request 'Fill mergedIds field and filter mergerels with dedup records actually created' (#500) from mergedids into beta 
Reviewed-on: D-Net/dnet-hadoop#500
 2024-10-28 13:43:09 +01:00
Giambattista Bloisi 56224e034a Fill the new mergedIds field when generating dedup records 
Filter out dedup records composed of invisible records only
Filter out mergerels that have not been used when creating the dedup record (ungrouping of cliques)
 2024-10-28 13:31:01 +01:00
Miriam Baglioni 5916346ba1 [TransformativeAgreement] fix to remove the file downloaded from a previous run of the workflow 2024-10-28 12:18:50 +01:00
Claudio Atzori e4abe55988 merged person_through_the_graph & code formatting 2024-10-28 11:01:49 +01:00
Claudio Atzori d71df6de19 Merge pull request 'affroNewModelonBeta' (#494) from affroNewModelonBeta into beta 
Reviewed-on: D-Net/dnet-hadoop#494
 2024-10-28 10:48:34 +01:00
Claudio Atzori 1cdcd07a7e Merge pull request 'dhp-schema upgrade & provision mapping 2' (#499) from beta_provision_alignment_9.0.0 into beta 
Reviewed-on: D-Net/dnet-hadoop#499
 2024-10-28 10:44:08 +01:00
Claudio Atzori 6fd50266f1 translate 'otherresearchproduct' into 'other' when setting the related record type 2024-10-28 10:42:46 +01:00
Claudio Atzori dffa376eb6 Merge pull request 'dhp-schema upgrade & provision mapping' (#498) from beta_provision_alignment_9.0.0 into beta 
Reviewed-on: D-Net/dnet-hadoop#498
 2024-10-28 10:03:24 +01:00
Claudio Atzori 32fa579b80 [graph provision] select the longest abstract 2024-10-28 10:03:02 +01:00
Claudio Atzori 67e37f41fb Merge pull request 'blacklist filtering moved before the cleanup phase in order to have case sensitive regex' (#485) from dedup_blacklist_fix into beta 
Reviewed-on: D-Net/dnet-hadoop#485
 2024-10-28 09:42:51 +01:00
Miriam Baglioni 0fb6af5586 Updated main pom dependency against dhp-schema, from 8.0.1 to 9.0.0. The new fields included in the updated schema module are populated by the Solr JSON payload mapping, which also limits the number of authors serialised to 200. 2024-10-25 16:28:50 +02:00
Claudio Atzori dcba5ad32a Merge pull request 'person_through_the_graph_newDevelopments' (#497) from person_through_the_graph_newDevelopments into person_through_the_graph 
Reviewed-on: D-Net/dnet-hadoop#497
 2024-10-25 10:20:40 +02:00
Claudio Atzori 46dbb62598 Merge pull request '#9839: include claimed affiliation relationships' (#476) from claim-orgs into beta 
Reviewed-on: D-Net/dnet-hadoop#476
 2024-10-25 10:12:59 +02:00
Claudio Atzori d3764265d5 Merge pull request '[dedup] avoid NPEs in the countryInference dedup utility' (#475) from dedup_countryInference_NPE into beta 
Reviewed-on: D-Net/dnet-hadoop#475
 2024-10-25 10:12:06 +02:00
Claudio Atzori 4a9aeb6238 Merge pull request '9126-impact-indicators-wf-optimisation' (#471) from 9126-impact-indicators-wf-optimisation into beta 
Reviewed-on: D-Net/dnet-hadoop#471
 2024-10-25 10:10:44 +02:00
Claudio Atzori 8172bee8c8 Merge pull request 'Minor fixes' (#496) from beta_fixes_oct into beta 
Reviewed-on: D-Net/dnet-hadoop#496
 2024-10-25 10:09:56 +02:00