D-Net
/
dnet-hadoop
15
4
Fork 10
Code Issues 14 Pull Requests 11 Projects 2 Releases 1 Wiki Activity

Master branch updates from beta October 2023 #358

Manually merged
claudio.atzori merged 0 commits from beta into master 2023-11-03 12:09:45 +01:00
Conversation 0 Commits - Files Changed -
claudio.atzori commented 2023-11-03 12:04:58 +01:00
Owner
Copy Link

This PR brings to the master branch the changes available from the beta branch at October 2023.

  • #340 import of POCI from Opencitations
  • #343 Integration with Software Heritage (result level PIDs)
  • #338 Run CC and RAM sequentieally in dhp-impact-indicators WF
  • #344 [DOIBoost] implemented relation to irish funder from a Json list from Crossref
  • #346 [UnresolvedEntities] changing in the creation of the unresolved entities to integrate FOS deeper levels
  • #349 [dedup] use common saveParquet and save methods to ensure outputs are compressed
  • #351 [dedup] FIX: GroupEntitiesSparkJob deletes whole graph outputPath instead of its temporary folder
  • #339 [dedup] Fix bug in conversion from dedup json model to Spark Dataset of Rows (instanceTypeMatch no longer working)
  • #333 [dedup] SparkPropagateRelation relations do not propagate deletedByInference and invisible
  • #334 [dedup] GroupEntities and DispatchEntites are now merged in GroupEntitiesSparkJob
  • #332 [stats] several changes in the indicators and monitor DBs
  • #342 [cleaning] Extending the coverage of the peer non-unknown refereed instances
  • #345 [cleaning] Fix cleaning of Pmid where parsing of numbers stopped at first not leading 0 (zero) character
This PR brings to the master branch the changes available from the beta branch at October 2023. * #340 import of POCI from Opencitations * #343 Integration with Software Heritage (result level PIDs) * #338 Run CC and RAM sequentieally in dhp-impact-indicators WF * #344 [DOIBoost] implemented relation to irish funder from a Json list from Crossref * #346 [UnresolvedEntities] changing in the creation of the unresolved entities to integrate FOS deeper levels * #349 [dedup] use common saveParquet and save methods to ensure outputs are compressed * #351 [dedup] FIX: GroupEntitiesSparkJob deletes whole graph outputPath instead of its temporary folder * #339 [dedup] Fix bug in conversion from dedup json model to Spark Dataset of Rows (instanceTypeMatch no longer working) * #333 [dedup] SparkPropagateRelation relations do not propagate deletedByInference and invisible * #334 [dedup] GroupEntities and DispatchEntites are now merged in GroupEntitiesSparkJob * #332 [stats] several changes in the indicators and monitor DBs * #342 [cleaning] Extending the coverage of the peer non-unknown refereed instances * #345 [cleaning] Fix cleaning of Pmid where parsing of numbers stopped at first not leading 0 (zero) character
claudio.atzori added 79 commits 2023-11-03 12:04:59 +01:00
163b2ee2a8 Changes 
1. Monitor updates
2. Bug fixes during copy to impala cluster
964c2f553e Changes in indicators step, monitor step 
- graduatedoctorates for observatory
- result_apc_affiliations table
- new indicators
	indi_is_funder_plan_s
	indi_funder_fairness
	indi_ris_fairness
	indi_funder_openess
	indi_ris_openess
	indi_funder_findable
	indi_ris_findable
	indi_is_project_result_after
- cast year to int in composite indicators
- new institutions
     -- Universidade Católica Portuguesa
     -- Iscte - Instituto Universitário de Lisboa
     -- Munster Technological University
     -- Cardiff University
     -- Leibniz Institute of Ecological Urban and Regional Development
2caaaec42d Include SparkCleanRelation logic in SparkPropagateRelation 
SparkPropagateRelation includes merge relations
Revised tests for SparkPropagateRelation
9ef971a146 Update step16-createIndicatorsTables.sql 
Fix int year for:
indi_org_openess_year
indi_org_fairness_year
indi_org_findable_year
489a082f04 Update step16-createIndicatorsTables.sql 
Change scripts for gold, hybrid, bronze indicators
17586f0ff8 Update step20-createMonitorDB.sql 
Add result_orcid table to monitor dbs
claudio.atzori manually merged commit 8c03c41d5d into master 2023-11-03 12:09:45 +01:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
bug

Something is not working

  
duplicate

This issue or pull request already exists

  
enhancement

New feature / refactoring

  
help wanted

Need some help

  
invalid

Something is wrong

  
question

More information is needed

  
RDGraph

EOSC Research Discovery Graph

  
RSAC

EOSC Research Software APIs and Connectors

  
wontfix

This won't be fixed

No Label
bug
duplicate
enhancement
help wanted
invalid
question
RDGraph
RSAC
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: D-Net/dnet-hadoop#358
No description provided.
Delete Branch "beta"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?