D-Net
/
dnet-hadoop
14
5
Fork 11
Code Issues 14 Pull Requests 12 Projects 2 Releases 2 Wiki Activity

OpenOrgs dedup and Integration with OpenAIRE Provision #102

Merged
claudio.atzori merged 12 commits from openorgswf into stable_ids 2021-03-29 16:41:09 +02:00
Conversation 1 Commits 12 Files Changed 49 +2282 -677
michele.debonis commented 2021-03-29 16:27:36 +02:00
Member
Copy Link

Implementation of the jobs and the workflow for the openorgs de-duplication that fills postgres tables for the openorgs portal.
Modification of existent workflows and jobs from the openaire side: organizations are treated separately and collected from the openorgs database.
Implementation of various tests for the above use cases
Implementation of a workflow for the creation of the raw organizations to be used by the openorgs wf
The query for the extractions of the records from the databases have been splitted into two different sections: one for the openaire use-case, one for the openorgs use case (they differ on the type of relations to collect from the db)

Implementation of the jobs and the workflow for the openorgs de-duplication that fills postgres tables for the openorgs portal. Modification of existent workflows and jobs from the openaire side: organizations are treated separately and collected from the openorgs database. Implementation of various tests for the above use cases Implementation of a workflow for the creation of the raw organizations to be used by the openorgs wf The query for the extractions of the records from the databases have been splitted into two different sections: one for the openaire use-case, one for the openorgs use case (they differ on the type of relations to collect from the db)
michele.debonis added the
enhancement
 label 2021-03-29 16:27:36 +02:00
claudio.atzori was assigned by michele.debonis 2021-03-29 16:27:36 +02:00
michele.debonis added 11 commits 2021-03-29 16:27:37 +02:00
michele.debonis added 1 commit 2021-03-29 16:39:42 +02:00
claudio.atzori commented 2021-03-29 16:41:04 +02:00
Owner
Copy Link

We're going to test the changes proponed in this PR on the BETA environment.

We're going to test the changes proponed in this PR on the BETA environment.
claudio.atzori merged commit 940556f6d3 into stable_ids 2021-03-29 16:41:09 +02:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
bug

Something is not working

  
duplicate

This issue or pull request already exists

  
enhancement

New feature / refactoring

  
help wanted

Need some help

  
invalid

Something is wrong

  
question

More information is needed

  
RDGraph

EOSC Research Discovery Graph

  
RSAC

EOSC Research Software APIs and Connectors

  
wontfix

This won't be fixed

No Label
bug
duplicate
enhancement
help wanted
invalid
question
RDGraph
RSAC
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
claudio.atzori
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: D-Net/dnet-hadoop#102
No description provided.
Delete Branch "openorgswf"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?