sequenceFilePath
the path to store the sequence file of the native metadata collected
mdStorePath
the path of the native mdstore
apiDescription
A json encoding of the API Description class
dataSourceInfo
A json encoding of the Datasource Info
identifierPath
An xpath to retrieve the metadata idnentifier for the generation of DNet Identifier
metadataEncoding
The type of the metadata XML/JSON
timestamp
The timestamp of the collection date
workflowId
The identifier of the workflow
Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]
${jobTracker}
${nameNode}
lib/dhp-collector-worker-1.0.0.jar
-p${sequenceFilePath}
-a${apiDescription}
-n${nameNode}
-rh${rmq_host}
-ru${rmq_user}
-rp${rmq_pwd}
-rr${rmq_report}
-ro${rmq_ongoing}
-usandro.labruzzo
-w${workflowId}
${jobTracker}
${nameNode}
yarn
cluster
GenerateNativeStoreSparkJob
eu.dnetlib.dhp.collection.GenerateNativeStoreSparkJob
dhp-aggregations-1.0.0-SNAPSHOT.jar
--num-executors 50 --conf spark.yarn.jars="hdfs://hadoop-rm1.garr-pa1.d4science.org:8020/user/oozie/share/lib/lib_20180405103059/spark2"
--encoding ${metadataEncoding}
--dateOfCollection ${timestamp}
--provenance ${dataSourceInfo}
--xpath${identifierPath}
--input${sequenceFilePath}
--output${mdStorePath}
-rh${rmq_host}
-ru${rmq_user}
-rp${rmq_pwd}
-rr${rmq_report}
-ro${rmq_ongoing}
-w${workflowId}