D-Net
/
dnet-hadoop
13
4
Fork 10
Code Issues 14 Pull Requests 7 Projects 2 Releases 1 Wiki Activity

Updated XMLIterator for splitting on different nodes #436

Merged
alessia.bardi merged 3 commits from dblp_collection_plugin into beta 2024-05-29 16:05:07 +02:00
Conversation 2 Commits 3 Files Changed 3 +101 -9
sandro.labruzzo commented 2024-05-21 13:51:34 +02:00
Owner
Copy Link

For DBLP, we need to split XML files using the FileGZIPCollector plugin based on different tag names. I've updated the XMLIterator to accept the splitOnElement parameter as a comma-separated list of values. This allows the XMLIterator to check if the tag name is one of the selected tags and split accordingly. If a single tag is provided, it operates in the standard way.

For DBLP, we need to split XML files using the FileGZIPCollector plugin based on different tag names. I've updated the XMLIterator to accept the splitOnElement parameter as a comma-separated list of values. This allows the XMLIterator to check if the tag name is one of the selected tags and split accordingly. If a single tag is provided, it operates in the standard way.
giambattista.bloisi was assigned by sandro.labruzzo 2024-05-21 13:51:34 +02:00
claudio.atzori was assigned by sandro.labruzzo 2024-05-21 13:51:34 +02:00
miriam.baglioni was assigned by sandro.labruzzo 2024-05-21 13:51:34 +02:00
sandro.labruzzo added 2 commits 2024-05-21 13:51:34 +02:00
schatz commented 2024-05-28 10:19:52 +02:00
Member
Copy Link

As far as I can test it, it seems to be working. @miriam.baglioni @claudio.atzori @giambattista.bloisi Can you also review the code and the functionality and merge it so that we proceed deploying it for the DBLP aggregation?

As far as I can test it, it seems to be working. @miriam.baglioni @claudio.atzori @giambattista.bloisi Can you also review the code and the functionality and merge it so that we proceed deploying it for the DBLP aggregation?
alessia.bardi commented 2024-05-29 16:04:36 +02:00
Owner
Copy Link

Good to go also from my point of view. I am merging the PR in beta.

Good to go also from my point of view. I am merging the PR in beta.
alessia.bardi added 1 commit 2024-05-29 16:04:42 +02:00
alessia.bardi merged commit eadfd8d71d into beta 2024-05-29 16:05:07 +02:00
alessia.bardi deleted branch dblp_collection_plugin 2024-05-29 16:05:07 +02:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
bug

Something is not working

  
duplicate

This issue or pull request already exists

  
enhancement

New feature / refactoring

  
help wanted

Need some help

  
invalid

Something is wrong

  
question

More information is needed

  
RDGraph

EOSC Research Discovery Graph

  
RSAC

EOSC Research Software APIs and Connectors

  
wontfix

This won't be fixed

No Label
bug
duplicate
enhancement
help wanted
invalid
question
RDGraph
RSAC
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
miriam.baglioni
claudio.atzori
giambattista.bloisi
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: D-Net/dnet-hadoop#436
No description provided.
Delete Branch "dblp_collection_plugin"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?