Pull Requests - dnet-hadoop - Gitea

New Pull Request

Fixes in Graph Provision

#434 by giambattista.bloisi was merged 2024-05-09 14:15:05 +02:00

beta_provision_relation

1 waiting review

Miscellaneous related to changes in MergeUtils

#429 by giambattista.bloisi was merged 2024-04-24 08:55:38 +02:00

misc_fixes_merge_entities

Refinements to PR #404: refactoring the Oaf records merge utilities into dhp-common

#422 by giambattista.bloisi was merged 2024-04-17 11:58:27 +02:00

revised_merge_logic

Enhance Dedup authors matching with algorithms used for ORCID enhancements (task 9690)

#419 by giambattista.bloisi was merged 2024-04-16 10:24:12 +02:00

dedup_authorsmatch_bytoken

Enrich authors with ORCID info using new matching algorithm

#398 by giambattista.bloisi was merged 2024-03-22 17:29:20 +01:00

new_orcid_enhancement

Revised procedure when converting json data into xml

#395 by giambattista.bloisi was merged 2024-02-28 10:38:55 +01:00

restiterator_xmlcleanup

Revised instance type comparisons in dedup phase

#393 by giambattista.bloisi was merged 2024-02-15 12:15:38 +01:00

revisedInstanceType

Set deletedbyinference =true to dedup aliases, created when a dedup in a previous build has been merged in a new dedup

#392 by giambattista.bloisi was merged 2024-02-08 15:29:30 +01:00

fix_dedupaliases_deletedbyinference

Use SparkSQL in place of Hive for executing step16-createIndicatorsTables.sql of stats update wf

#386 by giambattista.bloisi was merged 2024-01-29 09:12:00 +01:00

stats_with_spark_sql

Implements pivots table update oozie workflow

#376 by giambattista.bloisi was merged 2024-01-22 16:37:30 +01:00

update_pivots_table

Unify merge logic of entities in MergeUtils.class

#370 by giambattista.bloisi was merged 2024-03-22 10:53:15 +01:00

Improvements and refactoring in Dedup

#367 by giambattista.bloisi was merged 2024-01-11 11:24:07 +01:00

dedup_increasenumofblocks

FIX: GroupEntitiesSparkJob deletes whole graph outputPath instead of its temporary folder

#351 by giambattista.bloisi was merged 2023-10-17 08:40:24 +02:00

fix_consistency_missing_rels

OpenAIRE - DNet

use saveParquet method to ensure mergerel output is compressed

#348 by giambattista.bloisi was closed 2023-10-16 10:59:39 +02:00

fix_mergerels_notcompressed

OpenAIRE - DNet

Fix cleaning of Pmid where parsing of numbers stopped at first not leading 0 (zero) character

#345 by giambattista.bloisi was merged 2023-10-06 14:19:49 +02:00

fix_truncated_pmid

OpenAIRE - DNet

Fix bug in conversion from dedup json model to Spark Dataset of Rows (instanceTypeMatch no longer working)

#339 by giambattista.bloisi was merged 2023-10-02 11:34:20 +02:00

fix_dedupfailsonmatchinginstances

OpenAIRE - DNet

GroupEntities and DispatchEntites are now merged in GroupEntitiesSparkJob

#334 by giambattista.bloisi was merged 2023-10-02 11:25:28 +02:00

merge_entities_job

SparkPropagateRelation relations do not propagate deletedByInference and invisible

#333 by giambattista.bloisi was merged 2023-10-02 11:27:58 +02:00

consistency_keep_mergerels

Add sparkExecutorMemoryOverhead workflow config to set off-heap memory for Spark actions. If not explicitly set it is defaulted to 1Gb

#331 by giambattista.bloisi was merged 2023-08-29 16:31:37 +02:00

consistencywf_memoryoverhead_conf

Rewrite SparkPropagateRelation exploiting Dataframe API

#330 by giambattista.bloisi was merged 2023-08-29 10:47:15 +02:00

propagate_relation_rewrite

1 2