forked from D-Net/dnet-hadoop
Lampros Smyrnaios
abf0b69f29
- Use only hive commands in the Ocean Cluster, as the "impala-shell" will be removed from there to free-up resources. - Hugely improve the performance in every aspect of the copying process: a) speedup file-transferring and DB-deletion, b) eliminate permissions-assignment, "load" operations and "use $db" queries, c) retry only the "create view" statements and only as long as they depend on other non-created views, instead of trying to recreate all tables and views 5 consecutive times. - Add error-checks for the creation of tables and views. |
||
|---|---|---|
| .. | ||
| scripts | ||
| config-default.xml | ||
| copyDataToImpalaCluster.sh | ||
| finalizeImpalaCluster.sh | ||
| hist_snaps.sh | ||
| workflow.xml | ||