dnet-hadoop/dhp-workflows/dhp-stats-update
Lampros Smyrnaios d46b78b659 dhp-stats-update:
- Set Steps 2-7 and 9 to limit the amount of files generated by Spark, from 8000, down to 100, to improve file-transfer and querying performance.
- Allow the workflow to run up to Step10. The Step11 seems to have some issues even when using hive-action.
2024-04-18 15:40:27 +03:00
..
src/main/resources/eu/dnetlib/dhp/oa/graph/stats/oozie_app dhp-stats-update: 2024-04-18 15:40:27 +03:00
installProject.sh Update "dhp-stats-update" workflow to use "spark"-actions, instead of "hive" ones. 2024-04-15 16:22:40 +03:00
pom.xml Use SparkSQL in place of Hive for executing step16-createIndicatorsTables.sql of stats update wf 2024-01-26 21:56:55 +01:00
runOozieWorkfow.sh dhp-stats-update: 2024-04-17 14:03:59 +03:00