D-Net
/
dnet-hadoop
14
5
Fork 11
Code Issues 14 Pull Requests 12 Projects 2 Releases 2 Wiki Activity

FIX: GroupEntitiesSparkJob deletes whole graph outputPath instead of its temporary folder #351

Merged
claudio.atzori merged 2 commits from fix_consistency_missing_rels into beta 2023-10-17 08:40:24 +02:00
Conversation 0 Commits 2 Files Changed 1 +1 -1
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
dhp-common/src/main/java/eu/dnetlib/dhp/oa/merge/GroupEntitiesSparkJob.java
View File

@ -71,7 +71,7 @@ public class GroupEntitiesSparkJob {
conf,
isSparkSessionManaged,
spark -> {
HdfsSupport.remove(outputPath, spark.sparkContext().hadoopConfiguration());
HdfsSupport.remove(checkpointPath, spark.sparkContext().hadoopConfiguration());
groupEntities(spark, graphInputPath, checkpointPath, outputPath, filterInvisible);
});
}