1
0
Fork 0
dnet-hadoop/dhp-workflows/dhp-stats-hist-snaps
Lampros Smyrnaios abf0b69f29 Upgrade the copying operation to Impala Cluster:
- Use only hive commands in the Ocean Cluster, as the "impala-shell" will be removed from there to free-up resources.
- Hugely improve the performance in every aspect of the copying process: a) speedup file-transferring and DB-deletion, b) eliminate permissions-assignment, "load" operations and "use $db" queries, c) retry only the "create view" statements and only as long as they depend on other non-created views, instead of trying to recreate all tables and views 5 consecutive times.
- Add error-checks for the creation of tables and views.
2024-04-11 17:12:12 +03:00
..
src/main/resources/eu/dnetlib/dhp/oa/graph/stats-hist-snaps/oozie_app Upgrade the copying operation to Impala Cluster: 2024-04-11 17:12:12 +03:00
pom.xml Historical Snapshots Workflow 2024-01-04 15:11:04 +02:00