D-Net
/
dnet-hadoop
13
4
Fork 10
Code Issues 14 Pull Requests 8 Projects 2 Releases 1 Wiki Activity

Stats wf using hive only #274

Closed
dimitris.pierrakos wants to merge 0 commits from antonis.lempesis/dnet-hadoop:hive into beta
pull from: antonis.lempesis/dnet-hadoop:hive
merge into: D-Net:beta
Conversation 10 Commits - Files Changed - +928 -558
dimitris.pierrakos commented 2023-02-20 08:42:38 +01:00
Collaborator
Copy Link
No description provided.
dimitris.pierrakos added 30 commits 2023-02-20 08:42:39 +01:00
2a52a42169 Added 4 institutions: 
-University of Modena and Reggio Emilia
-Bilkent University
-Saints Cyril and Methodius University of Skopje
-University of Milan
2a4bf32d4c Merge branch 'hive' of https://code-repo.d4science.org/antonis.lempesis/dnet-hadoop into hive 
# Conflicts:
#	dhp-workflows/dhp-stats-update/src/main/resources/eu/dnetlib/dhp/oa/graph/stats/oozie_app/scripts/step10.sql
#	dhp-workflows/dhp-stats-update/src/main/resources/eu/dnetlib/dhp/oa/graph/stats/oozie_app/scripts/step13.sql
#	dhp-workflows/dhp-stats-update/src/main/resources/eu/dnetlib/dhp/oa/graph/stats/oozie_app/scripts/step14.sql
#	dhp-workflows/dhp-stats-update/src/main/resources/eu/dnetlib/dhp/oa/graph/stats/oozie_app/scripts/step16_1-definitions.sql
#	dhp-workflows/dhp-stats-update/src/main/resources/eu/dnetlib/dhp/oa/graph/stats/oozie_app/scripts/step7.sql
claudio.atzori commented 2023-02-22 10:21:37 +01:00
Owner
Copy Link

I see the changeset includes a new module dhp-monitor-update. Does this module defines a workflow that must be operated in the context of the various stats DB updates? If so it has to be included among the deployment specification, so please let me know.

I see the changeset includes a new module `dhp-monitor-update`. Does this module defines a workflow that must be operated in the context of the various stats DB updates? If so it has to be included among the deployment specification, so please let me know.
dimitris.pierrakos commented 2023-02-22 10:41:58 +01:00
Author
Collaborator
Copy Link

This is a module that is supposed to be executed independently of the stats wf and whenever a new organization should be added to the Monitor dashboards. Its execution would not affect any part of the stats wf.

This is a module that is supposed to be executed independently of the stats wf and whenever a new organization should be added to the Monitor dashboards. Its execution would not affect any part of the stats wf.
dimitris.pierrakos added 2 commits 2023-02-22 10:49:19 +01:00
claudio.atzori commented 2023-02-22 11:18:10 +01:00
Owner
Copy Link

This is a module that is supposed to be executed independently of the stats wf and whenever a new organization should be added to the Monitor dashboards. Its execution would not affect any part of the stats wf.

Do I understand correctly that it must NOT be run along with the dhp-stats-update workflow?

> This is a module that is supposed to be executed independently of the stats wf and whenever a new organization should be added to the Monitor dashboards. Its execution would not affect any part of the stats wf. Do I understand correctly that it must NOT be run along with the `dhp-stats-update` workflow?
dimitris.pierrakos commented 2023-02-22 11:33:15 +01:00
Author
Collaborator
Copy Link

This is a module that is supposed to be executed independently of the stats wf and whenever a new organization should be added to the Monitor dashboards. Its execution would not affect any part of the stats wf.

Do I understand correctly that it must NOT be run along with the dhp-stats-update workflow?

Yes, exactly.

> > This is a module that is supposed to be executed independently of the stats wf and whenever a new organization should be added to the Monitor dashboards. Its execution would not affect any part of the stats wf. > > Do I understand correctly that it must NOT be run along with the `dhp-stats-update` workflow? Yes, exactly.
claudio.atzori commented 2023-02-22 11:41:07 +01:00
Owner
Copy Link

Yes, exactly.

Perfect, thanks! Do you have an estimate for resolving the conflicts?

> Yes, exactly. Perfect, thanks! Do you have an estimate for resolving the conflicts?
dimitris.pierrakos added 10 commits 2023-02-22 15:58:21 +01:00
686580a220 - New Monitor DB workflow 
- New Organization added
973d78a4d6 Update step15_5.sql 
Added unpaywalls open access colors
98c34263ed Update step20-createMonitorDB.sql 
Add University of Cape Town organization
dimitris.pierrakos added 2 commits 2023-04-06 09:53:19 +02:00
dimitris.pierrakos added 1 commit 2023-04-07 08:23:05 +02:00
c85de8fa1f -Added Technological University Dublin 
-Added project_organization_contribution table
-Add   Delft University of Technology
claudio.atzori commented 2023-04-19 16:23:16 +02:00
Owner
Copy Link

@dimitris.pierrakos I still see some conflicts. Please consider to pull the latest changes from the upstream origin before we move on integrating the PR.

@dimitris.pierrakos I still see some conflicts. Please consider to pull the latest changes from the upstream origin before we move on integrating the PR.
miriam.baglioni commented 2023-04-28 18:57:48 +02:00
Member
Copy Link

@dimitris.pierrakos we need to start a new provision round on beta next week. Do you think you can fix the conflicts? If the PR cannot be integrated we need to postpone this integration.

@dimitris.pierrakos we need to start a new provision round on beta next week. Do you think you can fix the conflicts? If the PR cannot be integrated we need to postpone this integration.
dimitris.pierrakos commented 2023-04-28 19:03:13 +02:00
Author
Collaborator
Copy Link

@dimitris.pierrakos we need to start a new provision round on beta next week. Do you think you can fix the conflicts? If the PR cannot be integrated we need to postpone this integration.
@miriam.baglioni can you please check it again? I made some fixes

> @dimitris.pierrakos we need to start a new provision round on beta next week. Do you think you can fix the conflicts? If the PR cannot be integrated we need to postpone this integration. @miriam.baglioni can you please check it again? I made some fixes
claudio.atzori commented 2023-05-02 09:30:16 +02:00
Owner
Copy Link

@miriam.baglioni can you please check it again? I made some fixes

@dimitris.pierrakos the conflicts, when present, are visible in this very page, at the bottom of this conversation. Currently it lists just three files that you should check

  • dhp-workflows/dhp-monitor-update/src/main/resources/eu/dnetlib/dhp/oa/graph/monitor/oozie_app/monitor.sh
  • dhp-workflows/dhp-monitor-update/src/main/resources/eu/dnetlib/dhp/oa/graph/monitor/oozie_app/scripts/createMonitorDB.sql
  • dhp-workflows/dhp-monitor-update/src/main/resources/eu/dnetlib/dhp/oa/graph/monitor/oozie_app/workflow.xml
> @miriam.baglioni can you please check it again? I made some fixes @dimitris.pierrakos the conflicts, when present, are visible in this very page, at the bottom of this conversation. Currently it lists just three files that you should check * dhp-workflows/dhp-monitor-update/src/main/resources/eu/dnetlib/dhp/oa/graph/monitor/oozie_app/monitor.sh * dhp-workflows/dhp-monitor-update/src/main/resources/eu/dnetlib/dhp/oa/graph/monitor/oozie_app/scripts/createMonitorDB.sql * dhp-workflows/dhp-monitor-update/src/main/resources/eu/dnetlib/dhp/oa/graph/monitor/oozie_app/workflow.xml
dimitris.pierrakos commented 2023-05-02 10:24:40 +02:00
Author
Collaborator
Copy Link

@miriam.baglioni please use this PR

#283

Apologies for the mistake.

@miriam.baglioni please use this PR https://code-repo.d4science.org/D-Net/dnet-hadoop/pulls/283 Apologies for the mistake.

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
bug

Something is not working

  
duplicate

This issue or pull request already exists

  
enhancement

New feature / refactoring

  
help wanted

Need some help

  
invalid

Something is wrong

  
question

More information is needed

  
RDGraph

EOSC Research Discovery Graph

  
RSAC

EOSC Research Software APIs and Connectors

  
wontfix

This won't be fixed

No Label
bug
duplicate
enhancement
help wanted
invalid
question
RDGraph
RSAC
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: D-Net/dnet-hadoop#274
No description provided.
Delete Branch "antonis.lempesis/dnet-hadoop:hive"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?