mkallipo
/
affRo
1
1
Fork 0
Code Issues Pull Requests Packages Projects Releases 1 Wiki Activity

Oalex #13

Merged
mkallipo merged 15 commits from openaire-workflow-ready_2 into openaire-workflow-ready 2024-12-09 18:51:24 +01:00
Conversation 0 Commits 15 Files Changed 1 +1 -1
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit 0043e4051f - Show all commits

2
strings.py
View File

@ -69,7 +69,7 @@ exploded = spark.read.json(folder_path) \
col("rors").alias("OAlex"), col("rors").alias("OAlex"),
explode(col("raw_aff_string")).alias("affiliation") #this allows to split all the raw_aff_string and to parallelize better explode(col("raw_aff_string")).alias("affiliation") #this allows to split all the raw_aff_string and to parallelize better
) )
affs = explode \ affs = exploded \
.select("affiliation") \ .select("affiliation") \
.distinct() \ .distinct() \
.withColumn("Matchings", oalex_affro_udf(col("affiliation"))) .withColumn("Matchings", oalex_affro_udf(col("affiliation")))