D-Net
/
dnet-hadoop
14
5
Fork 11
Code Issues 14 Pull Requests 13 Projects 2 Releases 2 Wiki Activity

doidoost_dismiss #418

Merged
claudio.atzori merged 30 commits from doidoost_dismiss into beta 2024-04-17 12:01:14 +02:00
Conversation 0 Commits 30 Files Changed 34 +14712 -2185
sandro.labruzzo commented 2024-04-11 18:10:40 +02:00
Owner
Copy Link

This pull request aims to replace the monolithic DOIBoost datasource with separate datasources, each with its own dedicated aggregation workflow.
The new datasources are:

  • Crossref + Unpaywall
  • MAG
  • Crossref

The following changes have been made to the datasources:
*

  • Crossref: Filter constraints have been relaxed, and affiliation relations (~200k) have been added.
  • MAG: The mapping has been revised, and records without DOIs have been added. see documente here
This pull request aims to replace the monolithic DOIBoost datasource with separate datasources, each with its own dedicated aggregation workflow. The new datasources are: * Crossref + Unpaywall * MAG * Crossref The following changes have been made to the datasources: * * Crossref: Filter constraints have been relaxed, and affiliation relations (~200k) have been added. * MAG: The mapping has been revised, and records without DOIs have been added. see documente [here](https://docs.google.com/document/d/14AW5QLIkNZ4W4kzPO4OLzdC8OTPmJAKHbM4c--jFu3c/edit)
sandro.labruzzo added 26 commits 2024-04-11 18:10:40 +02:00
ad0e9aa80c added first part of refactoring of the code generating MAG, 
make it more readable using spark sql queries
c532831718 Moved Crossref Mapping on dhp-aggregations, 
refactored code, avoid to use utility for create part of the oaf defined in DOIBoostMappingUtils, used instead utility in OafMappingUtils
98dc042db5 mapping generated for MAG, 
missing generation of Organization Action set
claudio.atzori was assigned by sandro.labruzzo 2024-04-16 10:16:55 +02:00
giambattista.bloisi was assigned by sandro.labruzzo 2024-04-16 10:16:55 +02:00
sandro.labruzzo added 1 commit 2024-04-16 13:39:23 +02:00
miriam.baglioni added 2 commits 2024-04-16 15:27:12 +02:00
claudio.atzori added 1 commit 2024-04-17 12:01:02 +02:00
claudio.atzori merged commit b554c41cc7 into beta 2024-04-17 12:01:13 +02:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
bug

Something is not working

  
duplicate

This issue or pull request already exists

  
enhancement

New feature / refactoring

  
help wanted

Need some help

  
invalid

Something is wrong

  
question

More information is needed

  
RDGraph

EOSC Research Discovery Graph

  
RSAC

EOSC Research Software APIs and Connectors

  
wontfix

This won't be fixed

No Label
bug
duplicate
enhancement
help wanted
invalid
question
RDGraph
RSAC
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
giambattista.bloisi
claudio.atzori
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: D-Net/dnet-hadoop#418
No description provided.
Delete Branch "doidoost_dismiss"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?