D-Net
/
dnet-hadoop
15
4
Fork 10
Code Issues 14 Pull Requests 10 Projects 2 Releases 1 Wiki Activity

Various fixes in the stats wf #430

Open
antonis.lempesis wants to merge 10 commits from antonis.lempesis/dnet-hadoop:beta into beta
pull from: antonis.lempesis/dnet-hadoop:beta
merge into: D-Net:beta
Conversation 0 Commits 10 Files Changed 6 +139 -155
antonis.lempesis commented 2024-04-24 11:18:07 +02:00
Member
Copy Link
No description provided.
antonis.lempesis added 10 commits 2024-04-24 11:18:08 +02:00
49af2e5740 Miscellaneous updates to the copying operation to Impala Cluster: 
- Update the algorithm for creating views that depend on other views; overcome some bash-instabilities.
- Upon any error, fail the whole process, not just the current DB-creation, as those errors usually indicate a bug in the initial DB-creation, that should be fixed immediately.
- Enhance parallel-copy of large files by "hadoop distcp" command.
- Reduce the "invalidate metadata" commands to just the current DB's tables, in order to eliminate the general overhead on Impala.
- Show the number of tables and views in the logs.
- Fix some log-messages.
This pull request can be merged automatically.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
You can also view command line instructions.

Step 1:

From your project repository, check out a new branch and test the changes.
git checkout -b antonis.lempesis-beta beta
git pull beta

Step 2:

Merge the changes and update on Gitea.
git checkout beta
git merge --no-ff antonis.lempesis-beta
git push origin beta
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
bug

Something is not working

  
duplicate

This issue or pull request already exists

  
enhancement

New feature / refactoring

  
help wanted

Need some help

  
invalid

Something is wrong

  
question

More information is needed

  
RDGraph

EOSC Research Discovery Graph

  
RSAC

EOSC Research Software APIs and Connectors

  
wontfix

This won't be fixed

No Label
bug
duplicate
enhancement
help wanted
invalid
question
RDGraph
RSAC
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: D-Net/dnet-hadoop#430
No description provided.
Delete Branch "antonis.lempesis/dnet-hadoop:beta"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?