sourcePath
the source path
whitelist
the white list
allowedtypes
the allowed types
sparkDriverMemory
memory for driver process
sparkExecutorMemory
memory for individual executor
sparkExecutorCores
number of cores used by single executor
Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]
${jobTracker}
${nameNode}
yarn-cluster
cluster
CountryPropagation
eu.dnetlib.dhp.countrypropagation.SparkCountryPropagationJob
dhp-propagation-${projectVersion}.jar
--executor-memory ${sparkExecutorMemory}
--executor-cores ${sparkExecutorCores}
--driver-memory=${sparkDriverMemory}
--conf spark.extraListeners="com.cloudera.spark.lineage.NavigatorAppListener"
--conf spark.sql.queryExecutionListeners="com.cloudera.spark.lineage.NavigatorQueryListener"
-mt yarn-cluster
--sourcePath${sourcePath}
--whitelist${whitelist}
--allowedtypes${allowedtypes}