D-Net
/
dnet-hadoop
14
5
Fork 11
Code Issues 14 Pull Requests 12 Projects 2 Releases 2 Wiki Activity

8172_impact_indicators_workflow #284

Merged
miriam.baglioni merged 49 commits from 8172_impact_indicators_workflow into beta 2023-08-14 15:50:48 +02:00
Conversation 7 Commits 49 Files Changed 23 +2
1 changed files with 2 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit 3d69f33c84 - Show all commits

2
dhp-workflows/dhp-impact-indicators/src/main/resources/eu/dnetlib/dhp/oa/graph/impact_indicators/oozie_app/create_openaire_ranking_graph.py
View File

@ -131,8 +131,10 @@ cites_df = spark.read.json(graph_folder + "/relation")\
& (F.col('dataInfo.deletedbyinference') == "false")\
& (F.col('dataInfo.invisible') == "false"))\
.drop('dataInfo.deletedbyinference').drop('dataInfo.invisible')\
.drop('deletedbyinference').drop('invisible')\
.repartition(num_partitions, 'citing').drop('relClass')\
.withColumn('collected_lower', F.expr('transform(value, x -> lower(x))'))\
.drop('collectedfrom.value')\
.drop('value')\
.where(
(F.array_contains(F.col('collected_lower'), "opencitations"))