D-Net
/
dnet-hadoop
14
5
Fork 11
Code Issues 14 Pull Requests 12 Projects 2 Releases 2 Wiki Activity
4,050 Commits 99 Branches 20 Tags 658 MiB
Commit Graph

3541 Commits

Author SHA1 Message Date
Alessia Bardi 982bcc1e35 test wrid pid and record identifier 2022-09-23 12:06:06 +02:00
Miriam Baglioni 960cb861a0 refactoring 2022-09-23 11:14:04 +02:00
Claudio Atzori c42850328e fixed semantic (subreltype) for ServiceOrganization relations 2022-09-22 16:23:25 +02:00
Miriam Baglioni 33bb79459e Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta 2022-09-22 15:55:17 +02:00
dimitrispie dcd85f8cd7 - Synchronize indicators in stats-db with monitor-db 
- added new openorg id for Nanyang Technological University
- changed openorg id for University of Helsinki #8088 ticket
 2022-09-22 13:33:07 +03:00
Claudio Atzori e45ec15221 Merge branch 'beta' into clean_country 2022-09-19 11:34:02 +02:00
Claudio Atzori 26e1badded added instance.url syntactical validation, avoid creating multiple duplicated URLs 2022-09-19 11:19:10 +02:00
Miriam Baglioni 5240ac3d7b [EOSC Tag] remove addition of eosc context for result with eosc if guidelines set 2022-09-19 11:02:18 +02:00
Claudio Atzori 192215a18e merged from branch discard-non-wellformed 2022-09-19 10:17:10 +02:00
Claudio Atzori e370e940d8 [aggregator graph] save invalid records aside for further inspection 2022-09-16 14:06:28 +02:00
Claudio Atzori 465e941214 Merge pull request '[stats wf] Changes to indicators tables' (#244) from antonis.lempesis/dnet-hadoop:beta into beta 
Reviewed-on: #244
 2022-09-16 10:13:58 +02:00
Claudio Atzori 1e42d984e1 [aggregator graph] save invalid records aside for further inspection 2022-09-15 10:49:42 +02:00
Alessia Bardi 9e7ec4198f fixed test 2022-09-14 18:08:56 +02:00
Claudio Atzori c48f6e9c57 [aggregator graph] save invalid records aside for further inspection 2022-09-14 17:11:26 +02:00
dimitrispie 3bf3127251 Changes to monitor and indicator scripts 2022-09-14 16:36:19 +03:00
Claudio Atzori a0919ed495 [aggregator graph] save invalid records aside for further inspection 2022-09-14 13:27:39 +02:00
Alessia Bardi b99a011345 return empty Oaf list if record cannot be parsed 2022-09-13 11:51:55 +02:00
Alessia Bardi 27af5122d2 logs for non well formed XML files 2022-09-12 14:25:23 +02:00
Claudio Atzori ff6f789b6d code formatting 2022-09-09 15:16:31 +02:00
Claudio Atzori b5d6966c01 Merge branch 'beta' into clean_country 2022-09-09 12:20:19 +02:00
Claudio Atzori b5f7bd30be Merge branch 'beta' into clean_subjects 2022-09-09 12:20:04 +02:00
Alessia Bardi f14107ad77 Merge branch 'handle_as_instance_urls' of https://code-repo.d4science.org/D-Net/dnet-hadoop into handle_as_instance_urls 2022-09-09 12:17:19 +02:00
Alessia Bardi a539c6ccaf https for handle URLs 2022-09-09 12:16:28 +02:00
dimitrispie 71b069ca90 Changes to indicator and monitor scripts 2022-09-09 13:15:58 +03:00
Claudio Atzori 1203378441 Merge branch 'beta' into clean_subjects 2022-09-09 10:38:47 +02:00
Claudio Atzori 14dc909a14 Merge branch 'beta' into clean_country 2022-09-09 10:38:17 +02:00
Claudio Atzori 853c996fa2 Merge branch 'beta' into handle_as_instance_urls 2022-09-09 09:47:16 +02:00
Claudio Atzori a431e01383 Merge pull request 'orcid_multipleworks_download' (#242) from enrico.ottonello/dnet-hadoop:orcid_multipleworks_download into beta 
Reviewed-on: #242
 2022-09-09 08:45:02 +02:00
Alessia Bardi 9ef063d502 #7861#note-8 instance url from handle 2022-09-07 17:29:54 +03:00
Alessia Bardi 5c45d52af3 testing for RiuNet 2022-09-07 15:40:57 +03:00
dimitrispie 2b5f8c9c9a comment out duplicate table creation 2022-09-06 12:27:53 +03:00
Alessia Bardi a11eb38065 testing for RO-Hub 2022-09-02 16:07:36 +02:00
Enrico Ottonello bfdf2dc390 Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into orcid_multipleworks_download 2022-08-25 12:07:54 +02:00
Enrico Ottonello da1cf561e6 alignment with beta 2022-08-25 11:57:20 +02:00
Enrico Ottonello 27445ccdaa cleaned log 2022-08-25 11:56:14 +02:00
Claudio Atzori b7c387c21f cleaning of subjects: avoid duplicated subjects, prioritise collected vs inferred or other sources 2022-08-12 15:09:16 +02:00
Claudio Atzori adb526b0e1 Merge branch 'beta' into clean_subjects 2022-08-12 10:51:17 +02:00
Claudio Atzori cb7c07c54e [scholix] added step to create tar archive 2022-08-11 11:25:24 +02:00
Claudio Atzori 2aa16d0432 [scholix] fixed OpenCitation dump procedure 2022-08-10 17:39:29 +02:00
Miriam Baglioni 7dbdd4a0fe [Clean Country]changes related to #241 (comment) 2022-08-10 15:13:10 +02:00
Claudio Atzori 51ad93e545 [scholix] fixed OpenCitation dump procedure 2022-08-10 11:57:56 +02:00
Miriam Baglioni 62d2138806 [Clean Context] changed a bit the logic. Added the check not to have result hosted by a datasource of type institutional repository from NL. Added also the check that the country should have been included in the result via propagation for it to be removed 2022-08-08 14:10:47 +02:00
Claudio Atzori 3418ce50ac cleaning of subjects: perform the cleaning when the given value is equivalent to one of the terms in the vocabulary 2022-08-08 12:48:47 +02:00
Claudio Atzori a78028dabc Merge branch 'beta' into clean_subjects 2022-08-08 12:34:33 +02:00
Miriam Baglioni 390013a4b2 mergin with branch beta 2022-08-08 12:30:31 +02:00
Claudio Atzori 3937ff04de Merge branch 'beta' into tagEosc 2022-08-08 09:57:23 +02:00
Claudio Atzori a4815f6bec Merge branch 'beta' into clean_subjects 2022-08-05 16:57:03 +02:00
Claudio Atzori 29c4cde42e Merge branch 'clean_subjects' of https://code-repo.d4science.org/D-Net/dnet-hadoop into clean_subjects 2022-08-05 16:56:37 +02:00
Claudio Atzori 4eaa063b1f cleaning of subjects 2022-08-05 16:56:09 +02:00
Claudio Atzori 84598c7535 Merge pull request 'restored some collab indicators' (#240) from antonis.lempesis/dnet-hadoop:beta into beta 
Reviewed-on: #240
 2022-08-05 15:50:39 +02:00
Antonis Lempesis fcef5294e2 restored some collab indicators 2022-08-05 13:45:01 +03:00
Claudio Atzori 844f6eb465 Merge branch 'beta' into clean_subjects 2022-08-05 12:39:05 +02:00
Claudio Atzori 32cee1f619 WIP: cleaning of subjects 2022-08-05 12:32:08 +02:00
Claudio Atzori c1f2ffc53d Merge pull request 'commenting out the collab indicators because they still fail' (#237) from antonis.lempesis/dnet-hadoop:beta into beta 
Reviewed-on: #237
 2022-08-05 11:57:36 +02:00
Antonis Lempesis 227e10f4b3 commenting out the collab indicators because they still fail 2022-08-05 12:54:36 +03:00
Claudio Atzori 6c0fd9284b merge from beta 2022-08-05 10:42:53 +02:00
Claudio Atzori b78889a0ce WIP: cleaning of subjects 2022-08-05 09:11:37 +02:00
Miriam Baglioni a7a18d7630 [Graph Dump] removed code for the dump from the project. Fixed issues in tests when possible 2022-08-04 17:40:40 +02:00
Claudio Atzori 499826ead1 serialising field eoscifguidelines field in the Solr XML records 2022-08-04 12:40:48 +02:00
Claudio Atzori 27a91841e7 WIP: cleaning of subjects 2022-08-04 11:39:39 +02:00
Antonis Lempesis b09d7ddc74 fixed the datasourceOrganization relations 2022-08-03 12:26:50 +02:00
Claudio Atzori e62018e95d [aggregator graph] added more assertions in test 2022-08-03 12:26:05 +02:00
Claudio Atzori efd96e7e66 Merge pull request 'fixed the datasourceOrganization relations' (#233) from antonis.lempesis/dnet-hadoop:beta into beta 
Reviewed-on: #233
 2022-08-03 12:25:05 +02:00
Antonis Lempesis 8b0407d8ec fixed the datasourceOrganization relations 2022-08-03 12:26:59 +03:00
Claudio Atzori eb53b52f7c code formatting 2022-08-02 13:24:47 +02:00
Claudio Atzori 27681cf6bf Merge pull request '[stats wf] latest version of indicators + added FOS classification' (#232) from antonis.lempesis/dnet-hadoop:beta into beta 
Reviewed-on: #232
 2022-08-02 12:57:15 +02:00
Antonis Lempesis 1778d40c40 latest version of indicators 2022-08-02 13:39:34 +03:00
Claudio Atzori 209c7e9dab [datacite] avoid UnsupportedOperationException 2022-08-01 09:05:35 +02:00
Enrico Ottonello 64311b8be4 removed unuseful accumulator 2022-07-31 01:03:29 +02:00
Antonis Lempesis 9886fe87ec - Added FOS classification 
- Added extra orgs in monitor
- Fixed result-project and organization-project tables
 2022-07-29 16:34:50 +03:00
Claudio Atzori 92e48f12f7 [metadata collection] updated collector plugin name 2022-07-29 13:54:00 +02:00
Claudio Atzori f62c4e05cd code formatting 2022-07-29 11:56:01 +02:00
Claudio Atzori 0727f0ef48 [EOSC tag] avoid NPEs 2022-07-29 11:55:34 +02:00
Miriam Baglioni 3329b6ce6b [EOSC TAG] added fix for NPE on subjects 2022-07-29 10:54:20 +02:00
Claudio Atzori 1dd1e4fe3a extended test for mapping project_organization relations 2022-07-28 11:27:08 +02:00
Claudio Atzori 60e4fbd78b Merge branch 'beta' into project_organization_contribution 2022-07-28 10:15:43 +02:00
Claudio Atzori ed98a6d9d0 [Datacite mapping] include the older datacite prefixed OpenAIRE id among the originalId[] 2022-07-28 10:15:14 +02:00
Claudio Atzori 09ccc7b472 Merge branch 'beta' into project_organization_contribution 2022-07-28 09:49:59 +02:00
Sandro La Bruzzo 67525076ec fixed test, now it compiles after commit a6977197b3 2022-07-26 15:35:17 +02:00
Claudio Atzori 26104826c4 Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta 2022-07-26 14:34:29 +02:00
Claudio Atzori d43663d30f adapted RorActionSet test, it should not create parent/child rels 2022-07-25 17:54:10 +02:00
Miriam Baglioni 35bcd9422d [EOSC Context Tagging] removed not needed specification in path 2022-07-25 15:45:22 +02:00
Miriam Baglioni 1c82acb168 [EOSC Context Tagging] refactoring: moved EOSC IF tagging in package eosc under bulkTag 2022-07-25 14:26:39 +02:00
Miriam Baglioni 68cb637832 merge with branch beta 2022-07-25 14:24:25 +02:00
Miriam Baglioni 0172bab251 [EOSC Context Tagging] refactoring 2022-07-25 14:16:45 +02:00
Claudio Atzori 612b7a5530 Merge branch 'beta' into tagEosc 2022-07-25 14:12:59 +02:00
Claudio Atzori c3ede1b379 Merge branch 'beta' into pubmed_update 2022-07-25 14:10:22 +02:00
Miriam Baglioni 144c103b67 [EOSC Context Tagging] add check to avoid the insertion of the context if already present 2022-07-25 13:52:45 +02:00
Enrico Ottonello 657b0208a2 multiple works download (<=100) for single request 2022-07-25 12:37:39 +02:00
Miriam Baglioni d091866e48 [EOSC Context Tagging] refactoring 2022-07-25 11:12:22 +02:00
Miriam Baglioni 5968ec018d [Clean Country] modified workflow and added param file 2022-07-22 16:48:38 +02:00
Miriam Baglioni a12d28c644 [Clean Country] added logic not to remove country from result if it exist a hosting datasource with that country. Moreover the country will be removed only if added with propagation 2022-07-22 16:23:12 +02:00
Miriam Baglioni 2c933f1158 mergin with branch beta 2022-07-22 14:57:41 +02:00
Miriam Baglioni 06a95daf60 [EOSC context TAG] refactoring after compilation 2022-07-22 14:57:06 +02:00
Miriam Baglioni ffb0ce3fb9 mergin with branch beta 2022-07-22 14:55:55 +02:00
Miriam Baglioni 627332526b [EOSC context TAG] workflow start from reset_outputpath action 2022-07-22 14:55:11 +02:00
Miriam Baglioni 7a1c1b6f53 [EOSC context TAG] Add test class and resourcesK 2022-07-22 14:36:02 +02:00
Sandro La Bruzzo ddc414b258 fixed wrong json param 2022-07-22 09:43:15 +02:00
Miriam Baglioni 317a4a56ef [EOSC context TAG] first implementation of the logic to tag results imported from datasources registered in the EOSC 2022-07-21 17:37:48 +02:00
Miriam Baglioni 3be036f290 [EOSC TAG] refactoring after compilation 2022-07-21 14:45:43 +02:00