D-Net
/
dnet-hadoop
13
4
Fork 10
Code Issues 14 Pull Requests 7 Projects 2 Releases 1 Wiki Activity

Oozie workflow for cleancontext #216

Merged
claudio.atzori merged 7 commits from cleancontext into beta 2022-04-22 15:46:40 +02:00
Conversation 0 Commits 7 Files Changed 1 +8
1 changed files with 8 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit e0915061c2 - Show all commits

8
dhp-workflows/dhp-graph-mapper/src/main/resources/eu/dnetlib/dhp/oa/graph/clean/oozie_app/workflow.xml
View File

@ -17,6 +17,14 @@
<name>shouldCleanContext</name>
<description>true if the context have to be cleaned</description>
</property>
<property>
<name>contextId</name>
claudio.atzori commented 2022-04-22 11:46:50 +02:00
Review
Copy Link

It should be better to include a description for this parameter, to explain its purpose and if it possible to include multiple contentIds, how they should be formatted.

It should be better to include a description for this parameter, to explain its purpose and if it possible to include multiple contentIds, how they should be formatted.
miriam.baglioni commented 2022-04-22 15:15:28 +02:00
Review
Copy Link

This is just the first naive implementation of the context cleaning. I have no idea how it will be once done properly

This is just the first naive implementation of the context cleaning. I have no idea how it will be once done properly
claudio.atzori commented 2022-04-22 15:27:43 +02:00
Review
Copy Link

It might be the 1st naive implementation, but looking at the oozie workflow, it is not obvious what a parameter plays when it is not accompanied by any description.

It might be the 1st naive implementation, but looking at the oozie workflow, it is not obvious what a parameter plays when it is not accompanied by any description.
miriam.baglioni commented 2022-04-22 15:42:35 +02:00
Review
Copy Link

extended

extended
<value>sobigdata</value>
</property>
<property>
<name>verifyParam</name>
claudio.atzori commented 2022-04-22 11:47:22 +02:00
Outdated
Review
Copy Link

Missing description.

Missing description.
miriam.baglioni commented 2022-04-22 15:16:16 +02:00
Outdated
Review
Copy Link

Same holds as for the previous comment. Anyway if you think it is important to have the descriptions I will add them

Same holds as for the previous comment. Anyway if you think it is important to have the descriptions I will add them
claudio.atzori commented 2022-04-22 15:29:51 +02:00
Outdated
Review
Copy Link

I think it is. Again: this PR might just provide a first implementation, but the businness logic around these two parameters exists only in your head. To understand their role I'd need to open the actual job implementation and reverse engeener it. I would appreciate if you could add a description.

I think it is. Again: this PR might just provide a first implementation, but the businness logic around these two parameters exists only in your head. To understand their role I'd need to open the actual job implementation and reverse engeener it. I would appreciate if you could add a description.
<value>gcube </value>
</property>
<property>
<name>sparkDriverMemory</name>