[UnresolvedEntities] changing in the creation of the unresolved entities #346

miriam.baglioni · 2023-10-10T14:15:31+02:00

miriam.baglioni commented

2023-10-10 14:15:31 +02:00

This PR introduces the following changes:

extends the fos model to include the level4 and the scores for level3 and level4
removes the measures for BIP!Finder

When running the workflow, it assumes to receive the input parameter delimiter which states the char used to split the values in the csv. Instead, the Bip input path parameter should be removed.

This PR introduces the following changes: * extends the fos model to include the level4 and the scores for level3 and level4 * removes the measures for BIP!Finder When running the workflow, it assumes to receive the input parameter **delimiter** which states the char used to split the values in the csv. Instead, the **Bip input path** parameter should be removed.

miriam.baglioni added 2 commits 2023-10-10 14:15:32 +02:00

110ce4b40f extend the fos model to include the level4 and the scores for level3 and level4. removed bip indicators from the instance

a431b04814 leftover for the properties and removal of bipfinder

miriam.baglioni requested review from claudio.atzori 2023-10-10 14:15:48 +02:00

claudio.atzori reviewed 2023-10-10 15:02:52 +02:00

dhp-workflows/dhp-aggregation/src/main/java/eu/dnetlib/dhp/actionmanager/createunresolvedentities/model/FOSDataModel.java

						
				@ -11,21 +11,43 @@ public class FOSDataModel implements Serializable {

					private String doi;

					@CsvBindByPosition(position = 1)

				//    @CsvBindByName(column = "doi")

claudio.atzori commented

2023-10-10 15:02:52 +02:00

In case you preferred using the position - based annotations, then consider to remove the column - based ones instead of keeping them commented in the code.

claudio.atzori reviewed 2023-10-10 15:06:44 +02:00

claudio.atzori left a comment

Please try to check if the spark native CSV reading capabilities works fine with the FOS Dataset, e.g.

val path = "/user/claudio.atzori/data/fos_lvl34"
val d = spark.read.options(Map("delimiter"->",", "header"->"true", "nullValue"->"N/A", "inferSchema"->"true")).csv(path)

See more @ https://spark.apache.org/docs/2.4.6/sql-data-sources-load-save-functions.html

Please try to check if the spark native CSV reading capabilities works fine with the FOS Dataset, e.g. ``` val path = "/user/claudio.atzori/data/fos_lvl34" val d = spark.read.options(Map("delimiter"->",", "header"->"true", "nullValue"->"N/A", "inferSchema"->"true")).csv(path) ``` See more @ https://spark.apache.org/docs/2.4.6/sql-data-sources-load-save-functions.html

claudio.atzori added 1 commit 2023-10-10 15:07:23 +02:00

66064e99fe Merge branch 'beta' into fos

claudio.atzori merged commit ecea58a41c into beta

2023-10-10 15:10:22 +02:00

claudio.atzori referenced this issue from a commit

2023-10-10 15:10:22 +02:00

Merge pull request '[UnresolvedEntities] changing in the creation of the unresolved entities' (#346) from fos into beta

claudio.atzori deleted branch fos

2023-10-10 15:10:22 +02:00

claudio.atzori added this to the OpenAIRE - DNet project 2023-10-26 10:00:23 +02:00

claudio.atzori referenced this pull request

2023-11-03 12:04:58 +01:00