spyros
/
dnet-hadoop
forked from D-Net/dnet-hadoop
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Correct filtering for MAG records

This commit is contained in:
Ilias Kanellos 2023-05-12 12:55:43 +03:00
parent 5ddbb4ad10
commit 1788ac2d4d
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
dhp-workflows/dhp-impact-indicators/src/main/resources/eu/dnetlib/dhp/oa/graph/impact_indicators/oozie_app/create_openaire_ranking_graph.py
View File

@ -137,7 +137,7 @@ cites_df = spark.read.json(graph_folder + "/relation")\
.where( .where(
(F.array_contains(F.col('collected_lower'), "opencitations")) (F.array_contains(F.col('collected_lower'), "opencitations"))
| (F.array_contains(F.col('collected_lower'), "crossref")) | (F.array_contains(F.col('collected_lower'), "crossref"))
| (F.array_contains(F.col('collected_lower'), "mag")) | (F.array_contains(F.col('collected_lower'), "microsoft academic graph"))
).drop('collected_lower') ).drop('collected_lower')
# print ("Cited df has: " + str(cites_df.count()) + " entries") # print ("Cited df has: " + str(cites_df.count()) + " entries")