D-Net
/
dnet-hadoop
13
4
Fork 10
Code Issues 14 Pull Requests 9 Projects 2 Releases 1 Wiki Activity

[stats wf] indicators across stats dbs & updates in the org ids #248

Closed
dimitris.pierrakos wants to merge 1742 commits from beta into beta2master_sept_2022
pull from: beta
merge into: D-Net:beta2master_sept_2022
Conversation 3 Commits 1742 Files Changed 1525 +1 -3
1 changed files with 1 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit 5f651f2316 - Show all commits

4
dhp-workflows/dhp-graph-mapper/src/main/scala/eu/dnetlib/dhp/sx/graph/SparkConvertRDDtoDataset.scala
View File

@ -112,13 +112,11 @@ object SparkConvertRDDtoDataset {
"opencitations".equalsIgnoreCase(k.getValue) "opencitations".equalsIgnoreCase(k.getValue)
) )
) )
.filter(r => r.getRelClass != null && r.getRelClass.equalsIgnoreCase(filterRelation)) .filter(r => r.getSubRelType != null && r.getSubRelType.equalsIgnoreCase(filterRelation))
spark.createDataset(rddRelation).as[Relation].write.mode(SaveMode.Overwrite).save(s"$relPath") spark.createDataset(rddRelation).as[Relation].write.mode(SaveMode.Overwrite).save(s"$relPath")
} else { } else {
val relationSemanticFilter = List( val relationSemanticFilter = List(
// "cites",
// "iscitedby",
"merges", "merges",
"ismergedin", "ismergedin",
"HasAmongTopNSimilarDocuments", "HasAmongTopNSimilarDocuments",