D-Net
/
dnet-hadoop
15
4
Fork 10
Code Issues 14 Pull Requests 10 Projects 2 Releases 1 Wiki Activity

Graph raw mapper: map new information in Relation #65

New Issue
Closed
opened 2020-12-07 16:48:07 +01:00 by alessia.bardi · 5 comments
alessia.bardi commented 2020-12-07 16:48:07 +01:00
Owner
Copy Link

Due to https://issue.openaire.research-infrastructures.eu/issues/6226, Miriam is going to add to ODF and OAF records in the aggregator new information about the fact that a link to a project has been validated by a funder.

The oaf:project element will therefore contain two new information:

  • validated (true, false, empty) - the attribute might not exist
  • validationDate (a date) - the attribute might not exist

In https://issue.openaire.research-infrastructures.eu/issues/6227#note-2 I proposed the following:

<oaf:projectid validated="true" validationDate="2020-01-03">snsf________::125306</oaf:projectid>

But please sync with @miriam.baglioni on the final format of the field to process.

When validated is empty, false or does not exist, it can be skipped. The information we care about is when it is true.

If the same record has two links to the same project and only one is validated, then the link is validated (i.e. validated=true wins)

The target property in the Relation class have the same names and are already defined.

Please work on a dedicated branch that could be included only on beta first.

Due to https://issue.openaire.research-infrastructures.eu/issues/6226, Miriam is going to add to ODF and OAF records in the aggregator new information about the fact that a link to a project has been validated by a funder. The oaf:project element will therefore contain two new information: * validated (true, false, empty) - the attribute might not exist * validationDate (a date) - the attribute might not exist In https://issue.openaire.research-infrastructures.eu/issues/6227#note-2 I proposed the following: `<oaf:projectid validated="true" validationDate="2020-01-03">snsf________::125306</oaf:projectid>` But please sync with @miriam.baglioni on the final format of the field to process. When validated is empty, false or does not exist, it can be skipped. The information we care about is when it is true. If the same record has two links to the same project and only one is validated, then the link is validated (i.e. validated=true wins) The target property in the Relation class have the same names and are already defined. Please work on a dedicated branch that could be included only on beta first.
alessia.bardi added the
enhancement
 label 2020-12-07 16:48:07 +01:00
michele.artini was assigned by alessia.bardi 2020-12-07 16:48:07 +01:00
michele.artini commented 2020-12-09 10:50:25 +01:00
Member
Copy Link

The final format of the relation will be:

<oaf:projectid  validationDate=“2020-01-03">snsf________::125306</oaf:projectid>

because the boolean validated can be calculated as validationDate != null

The final format of the relation will be: ``` <oaf:projectid validationDate=“2020-01-03">snsf________::125306</oaf:projectid> ``` because the boolean validated can be calculated as validationDate != null
michele.artini commented 2020-12-09 11:28:48 +01:00
Member
Copy Link

I performd the PR from branch rel_project_validation.

I performd the PR from branch **rel_project_validation**.
miriam.baglioni commented 2020-12-09 15:05:04 +01:00
Member
Copy Link

Please, set for validated relations a trust higher than that for non validated relations.

Please, set for validated relations a trust higher than that for non validated relations.
michele.artini commented 2020-12-09 16:20:39 +01:00
Member
Copy Link

I set a trust = 0.999 for validated rels

I set a trust = 0.999 for validated rels
claudio.atzori commented 2020-12-09 19:02:14 +01:00
Owner
Copy Link

PR integrated on the master branch in 4705144918

PR integrated on the master branch in https://code-repo.d4science.org/D-Net/dnet-hadoop/commit/4705144918aa41dc6349e7b9ad357e8899afe670
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
master
beta
spark34-integration
dependency-revision
beta-release-1.2.5
rest-collector-plugin-with-retry
misc_fixes_merge_entities
WebCrowlBeta
WebCrowl
provision_memoryOverhead
stats_step16_fix
doidoost_dismiss
CrossrefFundersMap
taggingProjects
9647_datacite_affiliations
UsageStatsRecordDS
mergeutils
oaf_country_beta
index_records
ocnew
FOSNew
bulkTaggingPathMapExtention
transformativeagreement
new_orcid_enhancement
9559_DBLP_data
base_stats_job_deprecated
SWH_issue_377
import_orps_fix
spark_join_param_tuning
crossref_mapping_vocabularies
promote_actions_join_type_master
promote_actions_join_type
provision_community_api
enrichmentSingleStepFixed
fosPreparationBeta
resource_types
enrichmentSingleStep
oldPropagationOrganizationCommunity
beta_to_master_dicember2023
orcid_import
9078_xml_records_irish_tender
clean_license_publisher
bulkTag
SWH_integration
importpoci
8172_impact_indicators_workflow
dedup-with-dataframe-spark34
8876
master_july23
distinct_pids_from_openorgs_beta
propagationProjectThroughParentChils
fulltext_url_validation
removeTaggingCondition
ticket_8369
tweaking_spark_params
fc4e-rsac
doiboost_authormerger
beta_dedup_configuration
apc_affiliation
bulkTagRefactor
organizationToRepresentative
graph_cleaning_refactoring
scholix_flat_indexing
scholix_data_type_openaire
advConstraintsInBeta
doiboostMappingExtention
mag_citation_relation
h2020classification
doiboostFunderExtention
citations_monodirectional
compatibility_order
8232-mdstore-synch-improve
subjectPropagation
pubmed_to_production
cleanCountryOnMaster
graph_cleaning
deduptesting
horizontalConstraints
enrichment
scholix_to_solr
transformation_wf
discard-non-wellformed
removeDump
eosc_context_tagging
pubmed_update
doiboost_refactor
clean_context_master
monitoring
dump_new_funded_products
dump_delta_projects
dump
7096-fileGZip-collector-plugin
oaf_relation_mapping
validation
native_records_migration
instance_group_by_url
hostedByMap_update
hostedByMap_oastartdate
sygma_indexing
september-2023
dhp-1.2.4
archive/master_pre_stable_ids
archive/bipFinder_master_test
archive/islookup_timeout
dhp-1.2.3
dhp-1.2.2
dhp-1.2.1
dhp-1.2.0
dhp-1.1.7
dhp-1.1.6
archive/junit5
archive/dedupTest
dhp-1.1.5
dhp-1.0.4
dhp-1.0.3
dhp-1.0.2
1.0.1
Labels
Clear labels   
bug

Something is not working

  
duplicate

This issue or pull request already exists

  
enhancement

New feature / refactoring

  
help wanted

Need some help

  
invalid

Something is wrong

  
question

More information is needed

  
RDGraph

EOSC Research Discovery Graph

  
RSAC

EOSC Research Software APIs and Connectors

  
wontfix

This won't be fixed

No Label
bug
duplicate
enhancement
help wanted
invalid
question
RDGraph
RSAC
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
michele.artini
4 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: D-Net/dnet-hadoop#65
No description provided.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?