Jobs for constructing a new graph from an elder one adding only new results from crossref and datacite #524

Closed

giambattista.bloisi wants to merge 1 commits from incremental_graph into beta

pull from: incremental_graph

merge into: D-Net:beta

D-Net:main

D-Net:beta

D-Net:dblp_collector_plugin

D-Net:incremental_graph_main

D-Net:beta_jan2025

D-Net:propagateorcid

D-Net:COnnectSubCommunities

D-Net:csv_collector_plugins

D-Net:solr_v9

D-Net:affiliation-matching

D-Net:hive_graph_db

D-Net:cleaning_10215

D-Net:beta-ukripublication

D-Net:scholix_small_index

D-Net:Communities_patents

D-Net:crossref_mapping_improvement

D-Net:raid_actionset

D-Net:zenodo_dump_collection

D-Net:datacite_dump_import

D-Net:main_bulktag

D-Net:actionset_promote_sequential

D-Net:person_through_the_graph_relazioniattributate

D-Net:pubmed_fix

D-Net:dedup_new_comparators

D-Net:abtracts_guidelines4

D-Net:propagateorcid_publisher

D-Net:merge_resulttypes

D-Net:betaFixPerson

D-Net:k8s_orcidpropagation

D-Net:raw_affiliation_mapping

D-Net:mergedids

D-Net:beta_provision_alignment_9.0.0

D-Net:bestinstancetype

D-Net:person_through_the_graph

D-Net:person_through_the_graph_newDevelopments

D-Net:affroNewModelonBeta

D-Net:beta_fixes_oct

D-Net:dedup_local_test_implementation

D-Net:fix_decision_tree

D-Net:pid_cleaning

D-Net:affRoModelModificationOnmain

D-Net:beta2main_sept2024

D-Net:osfPreprints_plugin

D-Net:fix_missing_project_rels

D-Net:enrich_more_subject_bug

D-Net:gtr2Publications_plugin

D-Net:merge_by_id_fix

D-Net:dedup_blacklist_fix

D-Net:claim-orgs

D-Net:9126-impact-indicators-wf-optimisation

D-Net:AffiliationFromPublisher

D-Net:pubmed_aggregation_improvements

D-Net:betaAffiliationFromPublisherPages

D-Net:oozie_spark_params

D-Net:GraphAnnotation

D-Net:rest-collector-request-header-map2

D-Net:dedup_countryInference_NPE

D-Net:webCrawlLessBlackList

D-Net:fos_l1l2

D-Net:sdgnodoi

D-Net:person

D-Net:tagOrganization

D-Net:entity_contexts

D-Net:broker_orcid

D-Net:metadata_collection_java_upgrade

D-Net:research_fi_collector_plugin

D-Net:openorgs_optimization

D-Net:checkContextOnDatasourceAndProject

D-Net:affRoFromRawStringmain

D-Net:openorgs_fixes

D-Net:master

D-Net:9872-create-solr-collection-aliases

D-Net:json_payload

D-Net:pivotselectionbypid

D-Net:import_openorg_type

D-Net:irish-oaipmh-exporter

D-Net:fix_mergedcliquesort

D-Net:spark34-integration

D-Net:beta_indexing_May2024

D-Net:mag_only_doi

D-Net:rest-collector-plugin-with-retry

D-Net:dependency-revision

D-Net:beta-release-1.2.5

D-Net:misc_fixes_merge_entities

D-Net:WebCrowlBeta

D-Net:WebCrowl

D-Net:doidoost_dismiss

D-Net:CrossrefFundersMap

D-Net:taggingProjects

D-Net:9647_datacite_affiliations

D-Net:UsageStatsRecordDS

D-Net:mergeutils

D-Net:oaf_country_beta

D-Net:index_records

D-Net:9559_DBLP_data

D-Net:base_stats_job_deprecated

D-Net:SWH_issue_377

D-Net:spark_join_param_tuning

D-Net:crossref_mapping_vocabularies

D-Net:oldPropagationOrganizationCommunity

D-Net:orcid_import

D-Net:dedup-with-dataframe-spark34

D-Net:ticket_8369

D-Net:tweaking_spark_params

D-Net:fc4e-rsac

D-Net:doiboost_authormerger

D-Net:beta_dedup_configuration

D-Net:scholix_flat_indexing

D-Net:scholix_data_type_openaire

D-Net:subjectPropagation

D-Net:enrichment

D-Net:validation

Conversation 0 Commits 1 Files Changed 22 +1157 -93

giambattista.bloisi commented 2025-01-13 11:57:27 +01:00

Member

Copy Link

No description provided.

giambattista.bloisi added 1 commit 2025-01-13 11:57:29 +01:00

6f9dc2d3ca Implement new jobs for collecting data from latest graph on hive and deltas from oaf mdstores (datacite and crossref) ...

Optimized CopyHdfsOafSparkApplication

giambattista.bloisi closed this pull request 2025-01-13 11:59:55 +01:00

Pull request closed

This pull request cannot be reopened because the branch was deleted.

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

bug

Something is not working

  

duplicate

This issue or pull request already exists

  

enhancement

New feature / refactoring

  

help wanted

Need some help

  

invalid

Something is wrong

  

question

More information is needed

  

RDGraph

EOSC Research Discovery Graph

  

RSAC

EOSC Research Software APIs and Connectors

  

wontfix

This won't be fixed

No Label

bug

duplicate

enhancement

help wanted

invalid

question

RDGraph

RSAC

wontfix

Milestone

Clear milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

No Assignees

1 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: D-Net/dnet-hadoop#524

No description provided.