relationPath
the source path
mergeRelPath
the target path
sparkDriverMemory
memory for driver process
sparkExecutorMemory
memory for individual executor
Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]
${jobTracker}
${nameNode}
yarn-cluster
cluster
Propagate Dedup Relations
eu.dnetlib.dedup.SparkPropagateRelationsJob
dhp-dedup-${projectVersion}.jar
--executor-memory ${sparkExecutorMemory}
--driver-memory=${sparkDriverMemory}
--num-executors 100
--conf spark.yarn.jars="hdfs://hadoop-rm1.garr-pa1.d4science.org:8020/user/oozie/share/lib/lib_20180405103059/spark2"
-mtyarn-cluster
--mergeRelPath${mergeRelPath}
--relationPath${relationPath}