Action manager: subdirs inside graph table dirs #7

New Issue

przemyslaw.jacewicz · 2020-04-15T17:44:59+02:00

2020-04-15 17:44:59 +02:00

Overview

Sample runs of promote action workflow resulted in creation of subdirs inside graph table dirs, for example:

dataset
├─ dataset
│  ├─ part-00000
│  ├─ part-00001
│  └─ ...

Action sets used for sample runs did not contain all OAF types, especially Result was not present. Promotion of all graph tables was on, indicated by active_* flags set to true.

## Overview Sample runs of promote action workflow resulted in creation of subdirs inside graph table dirs, for example: ```bash dataset ├─ dataset │ ├─ part-00000 │ ├─ part-00001 │ └─ ... ``` Action sets used for sample runs did not contain all OAF types, especially `Result` was not present. Promotion of all graph tables was on, indicated by `active_*` flags set to true.

przemyslaw.jacewicz commented

2020-04-15 17:54:29 +02:00

The most possible solution is that Oozie distcp actions in workflows do not delete their target dirs using prepare section and when target dir exists distcp copies source dir as a nested dir inside target dir. This behavior does not seem to be documented well.

The most possible solution is that Oozie distcp actions in workflows do not delete their target dirs using `prepare` section and when target dir exists distcp copies source dir as a nested dir inside target dir. This behavior does not seem to be documented well.

przemyslaw.jacewicz referenced a pull request that will close this issue

2020-04-15 18:00:28 +02:00

Closes #7: subdirs inside graph table dirs #8

claudio.atzori referenced this issue from a commit

2020-04-16 14:38:47 +02:00

Merge pull request 'Closes #7: subdirs inside graph table dirs' (#8) from przemyslaw.jacewicz/dnet-hadoop:przemyslawjacewicz_7_distcp_configuration_fix into master Run the code from this PR in isolation and it worked fine. Thanks!

claudio.atzori closed this issue

2020-04-16 14:38:47 +02:00

claudio.atzori added the

bug

label 2020-07-27 18:02:46 +02:00

Sign in to join this conversation.