Labels Milestones

OpenOrgs dedup and Integration with OpenAIRE Provision #102

Merged

claudio.atzori merged 12 commits from openorgswf into stable_ids 3 years ago

Conversation 1 Commits 12 Files Changed 49

michele.debonis commented 3 years ago

Collaborator

Implementation of the jobs and the workflow for the openorgs de-duplication that fills postgres tables for the openorgs portal.
Modification of existent workflows and jobs from the openaire side: organizations are treated separately and collected from the openorgs database.
Implementation of various tests for the above use cases
Implementation of a workflow for the creation of the raw organizations to be used by the openorgs wf
The query for the extractions of the records from the databases have been splitted into two different sections: one for the openaire use-case, one for the openorgs use case (they differ on the type of relations to collect from the db)

Implementation of the jobs and the workflow for the openorgs de-duplication that fills postgres tables for the openorgs portal. Modification of existent workflows and jobs from the openaire side: organizations are treated separately and collected from the openorgs database. Implementation of various tests for the above use cases Implementation of a workflow for the creation of the raw organizations to be used by the openorgs wf The query for the extractions of the records from the databases have been splitted into two different sections: one for the openaire use-case, one for the openorgs use case (they differ on the type of relations to collect from the db)

michele.debonis added the

enhancement

label 3 years ago

claudio.atzori was assigned by michele.debonis 3 years ago

michele.debonis added 11 commits 3 years ago

1e1aab83e3 implementation of the raw wf for openorgs: still not complete, some functionalities are missing

8fea29177c refactoring, minor changes and implementation of the wf for openorgs with integration of organization phases into the scan wf

c7e2d5a59a minor changes

4b2124a18e implementation of the openorgs wfs, implementation of the raw_all wf to migrate openorgs db entities

1a85020572 bug fix in graph-mapper, changes in the implementation of the openorgs wf to create relations and populate openorgs db

98854b0124 minor changes

348b0ef921 bug fix, implementation of the workflow for the creation of raw_organizations (openorgs dedup), addition of the pid lists to the openorgs postgres db

5dfb66b0fa minor changes

28c1cdd132 merged stable_ids into openorgswf

2355cc4e9b minor changes and bug fix

f446580e9f code refactoring (useless classes and wf removed), implementation of the test for the openorgs dedup

michele.debonis added 1 commit 3 years ago

2709d08fc2 Merge branch 'stable_ids' into openorgswf

claudio.atzori commented 3 years ago

Owner

We're going to test the changes proponed in this PR on the BETA environment.

We're going to test the changes proponed in this PR on the BETA environment.

claudio.atzori merged commit 940556f6d3 into stable_ids 3 years ago

claudio.atzori referenced this issue from a commit 3 years ago

Merge pull request 'OpenOrgs dedup and Integration with OpenAIRE Provision' (#102) from openorgswf into stable_ids

The pull request has been merged as 940556f6d3.

You can also view command line instructions.

Step 1:

From your project repository, check out a new branch and test the changes.

git checkout -b openorgswf stable_ids

git pull origin openorgswf

Step 2:

Merge the changes and update on Gitea.

git checkout stable_ids

git merge --no-ff openorgswf

git push origin stable_ids

Sign in to join this conversation.

Reviewers

Request review

No reviewers

Labels

Apply labels

Clear labels   

bug

Something is not working   

duplicate

This issue or pull request already exists   

enhancement

New feature / refactoring   

help wanted

Need some help   

invalid

Something is wrong   

question

More information is needed   

RDGraph

EOSC Research Discovery Graph   

RSAC

EOSC Research Software APIs and Connectors   

wontfix

This won't be fixed

No Label

bug

duplicate

enhancement

help wanted

invalid

question

RDGraph

RSAC

wontfix

Milestone

Set milestone

Clear milestone

No items

No Milestone

Projects

Set Project

Clear projects

No project

Assignees

Assign users

Clear assignees

No Assignees

claudio.atzori

2 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: D-Net/dnet-hadoop#102

Write Preview

Loading…

Cancel

Save

There is no content yet.