dnet-hadoop/dhp-workflows/dhp-stats-monitor-update
Lampros Smyrnaios 49af2e5740 Miscellaneous updates to the copying operation to Impala Cluster:
- Update the algorithm for creating views that depend on other views; overcome some bash-instabilities.
- Upon any error, fail the whole process, not just the current DB-creation, as those errors usually indicate a bug in the initial DB-creation, that should be fixed immediately.
- Enhance parallel-copy of large files by "hadoop distcp" command.
- Reduce the "invalidate metadata" commands to just the current DB's tables, in order to eliminate the general overhead on Impala.
- Show the number of tables and views in the logs.
- Fix some log-messages.
2024-04-23 17:15:04 +03:00
..
src/main/resources/eu/dnetlib/dhp/oa/graph/stats-monitor/oozie_app Miscellaneous updates to the copying operation to Impala Cluster: 2024-04-23 17:15:04 +03:00
pom.xml included new stats* workflows in parent pom list of modules, code formatting 2024-03-26 10:42:10 +01:00