D-Net
/
dnet-hadoop
14
5
Fork 11
Code Issues 14 Pull Requests 13 Projects 2 Releases 2 Wiki Activity
5,596 Commits 111 Branches 20 Tags 660 MiB
Commit Graph

5596 Commits

Author SHA1 Message Date
Miriam Baglioni 86fb19a7f0 [orcipPropagation] refactoring after compilation and fixed issue in path for propagation constants in incremental module 2024-12-20 14:56:06 +01:00
Miriam Baglioni 38572266d6 merging with branch beta 2024-12-20 14:42:11 +01:00
Miriam Baglioni 071cc95afb [orcipPropagation] test to verify the merge for multiple enrichment from multiple sources. It covers also the check for persistency of other identifiers types 2024-12-20 14:41:09 +01:00
Miriam Baglioni 29611b2091 [orcipPropagation]rewritten in scala. generategraph again not abstract 2024-12-20 12:40:33 +01:00
Miriam Baglioni 1853da1e2c [orcipPropagation] moved from joinWith to join not lo loose the schema, and removed the field matched.col(id) since there is no more Seq(id) 2024-12-20 11:35:11 +01:00
Miriam Baglioni 7752d47776 [orcipPropagation]changes for merge for propagation 2024-12-20 10:20:32 +01:00
Claudio Atzori a6da42a2e8 Merge pull request 'Update Gtr2 plugin' (#518) from beta-ukripublication into beta 
Reviewed-on: #518
 2024-12-20 10:11:34 +01:00
Claudio Atzori 7ff4111357 Merge pull request 'ConnectSubCommunities' (#523) from COnnectSubCommunities into beta 
Reviewed-on: #523
 2024-12-20 10:11:13 +01:00
Miriam Baglioni 849b75593e resolved conflicts 2024-12-20 09:21:22 +01:00
Miriam Baglioni 2d45f125a7 [bulktag subcommunities] refactoring and addition of new properties 2024-12-20 09:06:55 +01:00
Miriam Baglioni a9ccd00483 [orcidPropagatio] - 2024-12-20 08:35:17 +01:00
Giambattista Bloisi 3ad3a56868 Merge pull request 'Implement new jobs for constructing the graph incrementally' (#522) from incremental_graph into beta 
Reviewed-on: #522
 2024-12-19 15:14:41 +01:00
Miriam Baglioni 3021dfda77 [orcidPropagatio] - 2024-12-19 15:14:09 +01:00
Giambattista Bloisi 85dced4ffb Implement new jobs for collecting data from latest graph on hive and deltas from oaf mdstores (datacite and crossref) 
Optimized CopyHdfsOafSparkApplication
 2024-12-19 14:37:48 +01:00
Miriam Baglioni ec4a90f669 [orcidPropagatio] added specific classid and classname for pid qualifier 2024-12-19 11:03:09 +01:00
Miriam Baglioni 345d69d11b [orcidPropagatio] changed the test to include parameter and changed the values for the new resources 2024-12-19 11:02:42 +01:00
Miriam Baglioni 60cfaf119b [orcidPropagatio] added specific orcid propagation classid and classname in the qualifier 2024-12-19 11:02:11 +01:00
Miriam Baglioni df5f1caa7a [orcidPropagatio] added classid and classname for qualifier of the pid 2024-12-19 11:01:45 +01:00
Miriam Baglioni d9be4a36d4 [orcidPropagatio] chenaged resources since the code assumes to have bidirectionality in the relations 2024-12-19 11:01:25 +01:00
Miriam Baglioni 668706a2e3 [orcidPropagatio] changed to avoid searching relations only within the same type of results 2024-12-19 11:00:55 +01:00
Miriam Baglioni 40c002b112 Merge remote-tracking branch 'origin/propagateorcid' into propagateorcid 
# Conflicts:
#	dhp-common/src/main/java/eu/dnetlib/dhp/common/enrichment/Constants.java
#	dhp-common/src/main/scala/eu/dnetlib/dhp/common/author/SparkEnrichWithOrcidAuthors.scala
#	dhp-common/src/main/scala/eu/dnetlib/dhp/utils/ORCIDAuthorEnricher.scala
#	dhp-workflows/dhp-enrichment/src/main/java/eu/dnetlib/dhp/PropagationConstant.java
#	dhp-workflows/dhp-enrichment/src/main/java/eu/dnetlib/dhp/orcidtoresultfromsemrel/OrcidAuthors.java
#	dhp-workflows/dhp-enrichment/src/main/java/eu/dnetlib/dhp/orcidtoresultfromsemrel/SparkPropagateOrcidAuthor.java
#	dhp-workflows/dhp-graph-mapper/src/main/scala/eu/dnetlib/dhp/enrich/orcid/SparkEnrichGraphWithOrcidAuthors.scala
 2024-12-18 09:23:21 +01:00
Giambattista Bloisi 71fe0374dc Revise propagation tests 2024-12-17 16:01:03 +01:00
Giambattista Bloisi d095b31ea8 [orcidenrichment] Fix lambda to avoid requiring serialization on enclosing class 2024-12-17 15:17:11 +01:00
Giambattista Bloisi 6260526fa1 [orcidenrichment] Fix imports and formatting 2024-12-17 15:17:11 +01:00
Giambattista Bloisi 64f4d7fb71 [orcidenrichment] When comparing authors manage the case of hyphenation and punctuations characters and normalizes utf strings 2024-12-17 15:17:11 +01:00
Giambattista Bloisi e03e8a39c0 [orcidenrichment] Do not match in case of ambiguity: two authors match and at least one of them has affiliation string 2024-12-17 15:17:11 +01:00
Miriam Baglioni fbc19ce4a8 [orcidenrichment] fixing issue 2024-12-17 15:17:11 +01:00
Miriam Baglioni 0a0f820dc7 [orcidenrichment] fixing issue 2024-12-17 15:17:11 +01:00
Miriam Baglioni f9531e0406 [orcidenrichment] refactoring 2024-12-17 15:17:11 +01:00
Miriam Baglioni 1b4bbb2691 [orcidenrichment] refactoring 2024-12-17 15:17:11 +01:00
Miriam Baglioni da9bbdede4 [orcidenrichment] refactoring 2024-12-17 15:17:11 +01:00
Miriam Baglioni eb83a34f64 [OrcidPropagation] alignemnt of property file with new parameters 2024-12-17 15:17:11 +01:00
Miriam Baglioni 0cae085786 [OrcidPropagation] new preparation step to use the authornamedisambiguation employed for orcid enrichment. 2024-12-17 15:17:11 +01:00
Giambattista Bloisi 43a9fe1ef4 Draft SparkPropagateOrcidAuthors 2024-12-17 15:17:11 +01:00
Giambattista Bloisi 36ca0b123e Move AuthorMatchers in dhp-common 2024-12-17 15:17:11 +01:00
sandro.labruzzo dccbcfd36c code formatted 2024-12-13 11:48:32 +01:00
sandro.labruzzo b039952d97 bug fixed on zenodo plugin 2024-12-13 10:43:27 +01:00
Miriam Baglioni 29a2a29666 Merge pull request '[research_fi] added plugin name to collectorplugins' (#519) from beta_researchfi into beta 
Reviewed-on: #519
 2024-12-12 09:12:05 +01:00
Miriam Baglioni 1b1fb9f1c2 [research_fi] added plugin name to collectorplugins 2024-12-11 16:38:02 +01:00
Miriam Baglioni ce22b1d536 [gtr2 plugin] changed to try not to die if one publication link point to the website of the project 2024-12-11 16:33:51 +01:00
Giambattista Bloisi 101d9e830d JsonListMatch do not lower the extracted strings 
Fix test configurations and assertions
 2024-12-11 15:59:13 +01:00
Miriam Baglioni 19a9bddab1 [gtr2 plugin] changed to try not to die if one publication link point to the website of the project 2024-12-10 16:26:24 +01:00
Miriam Baglioni 69dad7e2bf [gtr2 plugin] removed unused import 2024-12-10 14:17:34 +01:00
Miriam Baglioni 9657707ab0 [gtr2 plugin] changed according to the new apis endpoint and response 2024-12-10 14:15:38 +01:00
Sandro La Bruzzo dd6ed31383 Merge remote-tracking branch 'origin/beta' into beta 2024-12-06 14:23:58 +01:00
Sandro La Bruzzo 0d05006114 code formatted 2024-12-06 14:23:47 +01:00
Claudio Atzori e4b814b3f1 code formatting 2024-12-06 13:58:39 +01:00
Claudio Atzori 5c7f7fb3b8 Merge pull request 'Add Collector Plugin for Zenodo Dumps' (#516) from zenodo_dump_collection into beta 
Reviewed-on: #516
 2024-12-06 13:51:10 +01:00
Claudio Atzori 9e6b1f2f24 Merge pull request 'Communities_patents' (#514) from Communities_patents into beta 
Reviewed-on: #514
 2024-12-06 13:50:43 +01:00
Miriam Baglioni 666155bafa [communityfromsemrelpropagation] changed resource to have deletedbyinference = false. 2024-12-06 12:26:41 +01:00