projectFileURL
the url where to get the projects file
programmeFileURL
the url where to get the programme file
topicFileURL
the url where to get the topic file
outputPath
path where to store the action set
sheetName
the name of the sheet to read
Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]
eu.dnetlib.dhp.actionmanager.project.utils.ReadCSV
--hdfsNameNode${nameNode}
--fileURL${projectFileURL}
--hdfsPath${workingDir}/projects
--classForNameeu.dnetlib.dhp.actionmanager.project.utils.CSVProject
eu.dnetlib.dhp.actionmanager.project.utils.ReadCSV
--hdfsNameNode${nameNode}
--fileURL${programmeFileURL}
--hdfsPath${workingDir}/programme
--classForNameeu.dnetlib.dhp.actionmanager.project.utils.CSVProgramme
eu.dnetlib.dhp.actionmanager.project.utils.ReadExcel
--hdfsNameNode${nameNode}
--fileURL${topicFileURL}
--hdfsPath${workingDir}/topic
--sheetName${sheetName}
--classForNameeu.dnetlib.dhp.actionmanager.project.utils.EXCELTopic
eu.dnetlib.dhp.actionmanager.project.ReadProjectsFromDB
--hdfsPath${workingDir}/dbProjects
--hdfsNameNode${nameNode}
--postgresUrl${postgresURL}
--postgresUser${postgresUser}
--postgresPassword${postgresPassword}
yarn
cluster
PrepareProgramme
eu.dnetlib.dhp.actionmanager.project.PrepareProgramme
dhp-aggregation-${projectVersion}.jar
--executor-cores=${sparkExecutorCores}
--executor-memory=${sparkExecutorMemory}
--driver-memory=${sparkDriverMemory}
--conf spark.extraListeners=${spark2ExtraListeners}
--conf spark.sql.queryExecutionListeners=${spark2SqlQueryExecutionListeners}
--conf spark.yarn.historyServer.address=${spark2YarnHistoryServerAddress}
--conf spark.eventLog.dir=${nameNode}${spark2EventLogDir}
--conf spark.sql.shuffle.partitions=3840
--programmePath${workingDir}/programme
--outputPath${workingDir}/preparedProgramme
yarn
cluster
PrepareProjects
eu.dnetlib.dhp.actionmanager.project.PrepareProjects
dhp-aggregation-${projectVersion}.jar
--executor-cores=${sparkExecutorCores}
--executor-memory=${sparkExecutorMemory}
--driver-memory=${sparkDriverMemory}
--conf spark.extraListeners=${spark2ExtraListeners}
--conf spark.sql.queryExecutionListeners=${spark2SqlQueryExecutionListeners}
--conf spark.yarn.historyServer.address=${spark2YarnHistoryServerAddress}
--conf spark.eventLog.dir=${nameNode}${spark2EventLogDir}
--conf spark.sql.shuffle.partitions=3840
--projectPath${workingDir}/projects
--outputPath${workingDir}/preparedProjects
--dbProjectPath${workingDir}/dbProjects
yarn
cluster
ProjectProgrammeAS
eu.dnetlib.dhp.actionmanager.project.SparkAtomicActionJob
dhp-aggregation-${projectVersion}.jar
--executor-cores=${sparkExecutorCores}
--executor-memory=${sparkExecutorMemory}
--driver-memory=${sparkDriverMemory}
--conf spark.extraListeners=${spark2ExtraListeners}
--conf spark.sql.queryExecutionListeners=${spark2SqlQueryExecutionListeners}
--conf spark.yarn.historyServer.address=${spark2YarnHistoryServerAddress}
--conf spark.eventLog.dir=${nameNode}${spark2EventLogDir}
--conf spark.sql.shuffle.partitions=3840
--projectPath${workingDir}/preparedProjects
--programmePath${workingDir}/preparedProgramme
--topicPath${workingDir}/topic
--outputPath${outputPath}