oct-update #1

Merged

sandro.labruzzo merged 42 commits from oct-update into master 2024-11-18 10:43:07 +01:00

Conversation 0 Commits 42 Files Changed 33 +1425 -231

sandro.labruzzo commented 2024-11-18 10:42:15 +01:00

Owner

Copy Link

Apply oct-update

Apply oct-update

sandro.labruzzo added 42 commits 2024-11-18 10:42:16 +01:00

7528675590 update version of minio

85cf6eeb1a fixed bucket creation updated tenant yaml

35c44845d2 added creation of bucket using variables

ed3422673f added git variable for airflow module

07ce192207 added workflow test

b8bf21f8e5 fixed import

6b555b8f6e added workflow test

dcd2efd3b4 added workflow test

d1afcd4395 fixed import

df6e23666e fix

412e008df7 Add untar task

c3ba29e4c5 Add dagutils

0fcabed2ae change dag name

8da265f018 add utils in the parent folder

bf7c9e2dce revert some changes

448bb924ab add test dedup task

ba3f351736 print existing files

c5f4263061 update spark-version

48f688cda9 add deps jar

0a2956d81f reduce executor cores

fa90a9dbe0 Update spark operator version

ae89274ce4 implemente whole scan pipeline

c6fbfd3f0a Remove numpartitions argument where not needed

034a01542a Implement consistency workflow

df46c8c65f Added ORCID enrichment workflows

131f6e5592 enable dynamic allocation

aae37058f7 Increase memory

73e78d6877 Add workflow with all graph construction steps

15ba3cf202 Provide paths as dag configuration parameters

1bd836b88a Provide paths as dag configuration parameters

e92ef9fbcb enable ingressUrlFormat

e64b3ec6b1 removed test dir

b3d7dda0c1 DAG to build the graph from a delta

6c25db9ac2 Clean DAG

7af06fbda5 Clean DAG

b01331d4d0 Clean DAG

9058dbb957 Clean DAG

004be2e97f Add resolvereletion step

c0788fcd10 Add resolvereletion step

ac2fbbb9f9 Add jupyterhub

13ac9767c6 Orcid propagation step

9e69ded5ef Copy relations in ORCID enrichment DAG

sandro.labruzzo merged commit 3e5781e52e into master 2024-11-18 10:43:07 +01:00

sandro.labruzzo referenced this issue from a commit 2024-11-18 10:43:08 +01:00

Merge pull request 'oct-update' (#1) from oct-update into master

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No Label

Milestone

Clear milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

No Assignees

1 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: D-Net/code-infrastructure-lab#1

No description provided.