sequenceFilePath
the path to store the sequence file of the native metadata collected
mdStorePath
the path of the native mdstore
apiDescription
A json encoding of the API Description class
dataSourceInfo
A json encoding of the Datasource Info
identifierPath
An xpath to retrieve the metadata idnentifier for the generation of DNet Identifier
metadataEncoding
The type of the metadata XML/JSON
timestamp
The timestamp of the collection date
Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]
${jobTracker}
${nameNode}
lib/dhp-collector-worker-1.0.0.jar
${sequenceFilePath}
${apiDescription}
${nameNode}
${jobTracker}
${nameNode}
yarn
cluster
MDBuilder
eu.dnetlib.dhp.collection.GenerateNativeStoreSparkJob
dhp-aggregations-1.0.0-SNAPSHOT.jar
--num-executors 50 --conf spark.yarn.jars="hdfs://hadoop-rm1.garr-pa1.d4science.org:8020/user/oozie/share/lib/lib_20180405103059/spark2"
--encoding ${metadataEncoding}
--dateOfCollection ${timestamp}
--provenance ${dataSourceInfo}
--xpath${identifierPath}
--input${sequenceFilePath}
--output${mdStorePath}