 Miriam Baglioni
|
016337a0f9
|
Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta
|
2023-02-16 15:54:59 +01:00 |
 Claudio Atzori
|
9a03f71db1
|
code formatting
|
2023-02-13 16:25:47 +01:00 |
|
Miriam Baglioni
|
32870339f5
|
refactoring after compile
|
2023-02-13 13:06:48 +01:00 |
|
Miriam Baglioni
|
7184cc0804
|
[FoS] added check for null on level1 subject
|
2023-02-13 13:03:49 +01:00 |
|
Miriam Baglioni
|
5cf902a2b0
|
[UsageCount] changed query to make the sum be computed via sql instead of grouping
|
2023-02-10 16:16:37 +01:00 |
|
Miriam Baglioni
|
f803530df6
|
[UsageCount] fixed query
|
2023-02-10 15:50:56 +01:00 |
|
Miriam Baglioni
|
7473093c84
|
[FoS] changed the default separator from comma to tab to solve the issue in subject value split
|
2023-02-10 15:34:52 +01:00 |
|
Miriam Baglioni
|
bb5bba51b3
|
[UsageCount] extended test
|
2023-02-09 19:08:30 +01:00 |
|
Miriam Baglioni
|
85e53fad00
|
[UsageCount] addition of usagecount for Projects and datasources. Extention of the action set created for the results with new entities for projects and datasources. Extention of the resource set and modification of the testing class
|
2023-02-09 18:59:45 +01:00 |
|
Claudio Atzori
|
f86e19b282
|
code formatting
|
2023-01-11 09:53:19 +01:00 |
 Michele Artini
|
7b7520850b
|
fixed an invalid char
|
2023-01-11 09:22:18 +01:00 |
|
Michele Artini
|
4953ae5649
|
fixed an invalid char
|
2023-01-11 08:35:53 +01:00 |
Sandro La Bruzzo
|
3c9826f186
|
updated lines function to it's implementation linesWithSeparators.map(l => l.stripLineEnd) in this way we force scala plugin compiler to consider this pipeline scala code and not java.string.lines() pipeline
|
2022-12-21 11:21:17 +01:00 |
|
Sandro La Bruzzo
|
91c70b15a5
|
updated lines function to it's implementation linesWithSeparators.map(l => l.stripLineEnd) in this way we force scala plugin compiler to consider this pipeline scala code and not java.string.lines() pipeline
|
2022-12-21 11:14:42 +01:00 |
 Sandro La Bruzzo
|
72f0d88d6c
|
formatted code
|
2022-10-19 14:18:42 +02:00 |
|
Sandro La Bruzzo
|
a1f94530a3
|
added documentation
|
2022-10-13 11:47:11 +02:00 |
|
Claudio Atzori
|
f3f7604e6c
|
trying to fix a test that fails only on Jenkins
|
2022-09-27 15:21:37 +02:00 |
|
Claudio Atzori
|
27a91841e7
|
WIP: cleaning of subjects
|
2022-08-04 11:39:39 +02:00 |
|
Claudio Atzori
|
eb53b52f7c
|
code formatting
|
2022-08-02 13:24:47 +02:00 |
|
Claudio Atzori
|
209c7e9dab
|
[datacite] avoid UnsupportedOperationException
|
2022-08-01 09:05:35 +02:00 |
|
Claudio Atzori
|
92e48f12f7
|
[metadata collection] updated collector plugin name
|
2022-07-29 13:54:00 +02:00 |
|
Claudio Atzori
|
f62c4e05cd
|
code formatting
|
2022-07-29 11:56:01 +02:00 |
|
Claudio Atzori
|
ed98a6d9d0
|
[Datacite mapping] include the older datacite prefixed OpenAIRE id among the originalId[]
|
2022-07-28 10:15:14 +02:00 |
|
Sandro La Bruzzo
|
67525076ec
|
fixed test, now it compiles after commit a6977197b3
|
2022-07-26 15:35:17 +02:00 |
|
Claudio Atzori
|
d43663d30f
|
adapted RorActionSet test, it should not create parent/child rels
|
2022-07-25 17:54:10 +02:00 |
|
Claudio Atzori
|
c3ede1b379
|
Merge branch 'beta' into pubmed_update
|
2022-07-25 14:10:22 +02:00 |
|
Claudio Atzori
|
1138b2ac8e
|
code formatting
|
2022-07-19 14:15:49 +02:00 |
|
Sandro La Bruzzo
|
00168303db
|
Added unit test to verify the generation in the OriginalID the old openaire Identifier generated by OAI
|
2022-07-14 10:19:59 +02:00 |
|
Sandro La Bruzzo
|
0a4f4d98fa
|
added PMCId to PmArticle
|
2022-07-13 15:27:17 +02:00 |
|
Claudio Atzori
|
0cb1c70788
|
code formatting
|
2022-07-01 10:44:08 +02:00 |
|
Claudio Atzori
|
929b145130
|
code formatting
|
2022-06-21 23:07:06 +02:00 |
|
Claudio Atzori
|
06b5533d4c
|
Merge branch 'beta' into 7096-fileGZip-collector-plugin
|
2022-06-16 09:22:16 +02:00 |
 Alessia Bardi
|
88d531dc91
|
exclude FAIRsharing records from Datacite
|
2022-06-13 16:17:17 +02:00 |
|
Claudio Atzori
|
b8cda65487
|
code formatting
|
2022-06-13 09:20:03 +02:00 |
|
Michele Artini
|
634869ce95
|
deleted hierarchical rels from ror action set
|
2022-06-13 09:12:21 +02:00 |
|
Michele Artini
|
b94a791bc5
|
unit tests to transform cnr explora
|
2022-06-09 12:25:34 +02:00 |
|
Claudio Atzori
|
d098ad0d93
|
[hb patch] updated map
|
2022-05-16 15:54:04 +02:00 |
|
Claudio Atzori
|
1dda11e031
|
[hb patch] updated map
|
2022-05-16 15:53:27 +02:00 |
|
Claudio Atzori
|
8dd5517548
|
code formatting
|
2022-05-16 14:35:24 +02:00 |
|
Michele Artini
|
46c07e0724
|
deleted hierarchical rels from ror action set
|
2022-05-16 09:39:54 +02:00 |
|
Miriam Baglioni
|
89657a0b78
|
[UsageCount] refactoring
|
2022-05-09 14:43:27 +02:00 |
|
Miriam Baglioni
|
a056f59c6e
|
[UsageCount] make it as an action set as it should be, plus changed the test to make them work as well now
|
2022-05-09 12:51:35 +02:00 |
Serafeim Chatzopoulos
|
623f7be26d
|
Fix reading files from HDFS in FileCollector & FileGZipCollector plugins
|
2022-04-28 16:31:11 +03:00 |
|
Claudio Atzori
|
30105f0722
|
Merge branch 'beta' into 7096-fileGZip-collector-plugin
|
2022-04-22 11:22:21 +02:00 |
|
Miriam Baglioni
|
20de75ca64
|
[Measures] removed typo
|
2022-04-21 12:14:03 +02:00 |
|
Miriam Baglioni
|
b61efd613b
|
[Measures] addressed comments in the PR
|
2022-04-21 12:09:37 +02:00 |
|
Claudio Atzori
|
eabb40fccc
|
Merge branch 'beta' into 7096-fileGZip-collector-plugin
|
2022-04-21 11:42:43 +02:00 |
|
Miriam Baglioni
|
c304657d91
|
[Measures] put the logic in common, no need to change the schema
|
2022-04-21 11:27:26 +02:00 |
|
Miriam Baglioni
|
5295effc96
|
[Measures] fixed issue
|
2022-04-20 16:20:40 +02:00 |
|
Miriam Baglioni
|
5feae77937
|
[Measures] last changes to accomodate tests
|
2022-04-20 15:13:09 +02:00 |
|
Miriam Baglioni
|
869407c6e2
|
[Measures] added new measure (usagecounts) as action set. Measure added at the level of the result. Ref #7587
|
2022-04-20 14:02:05 +02:00 |
|
Serafeim Chatzopoulos
|
d0b84d3297
|
Add FileCollectorPlugin and respective test
|
2022-04-07 15:06:38 +03:00 |
|
Serafeim Chatzopoulos
|
bc1bf55507
|
Add AbstractSplittedRecordPlugin
|
2022-04-07 14:33:04 +03:00 |
|
Claudio Atzori
|
c26222623f
|
[maven-release-plugin] prepare for next development iteration
|
2022-04-07 13:32:22 +02:00 |
|
Claudio Atzori
|
86585a6b27
|
[maven-release-plugin] prepare release dhp-1.2.4
|
2022-04-07 13:32:19 +02:00 |
|
Claudio Atzori
|
ad85d88eaf
|
[maven-release-plugin] rollback the release of dhp-1.2.4
|
2022-04-07 13:28:35 +02:00 |
|
Claudio Atzori
|
598e11dfd7
|
[maven-release-plugin] prepare for next development iteration
|
2022-04-07 13:27:02 +02:00 |
|
Claudio Atzori
|
db3d9877a5
|
[maven-release-plugin] prepare release dhp-1.2.4
|
2022-04-07 13:26:58 +02:00 |
|
Claudio Atzori
|
3bba6d6e38
|
[maven-release-plugin] rollback the release of dhp-1.2.4
|
2022-04-07 12:23:17 +02:00 |
|
Claudio Atzori
|
2ac2d928bd
|
[maven-release-plugin] prepare for next development iteration
|
2022-04-07 12:18:47 +02:00 |
|
Claudio Atzori
|
85bc722ff4
|
[maven-release-plugin] prepare release dhp-1.2.4
|
2022-04-07 12:18:43 +02:00 |
|
Claudio Atzori
|
bc05b6168a
|
[maven-release-plugin] rollback the release of dhp-1.2.4
|
2022-04-07 11:49:06 +02:00 |
|
Claudio Atzori
|
505420fd61
|
[maven-release-plugin] prepare for next development iteration
|
2022-04-07 11:34:06 +02:00 |
|
Claudio Atzori
|
66e718981e
|
[maven-release-plugin] prepare release dhp-1.2.4
|
2022-04-07 11:34:02 +02:00 |
|
Serafeim Chatzopoulos
|
e612489670
|
Add fileGZip collector plugin and respective test
|
2022-04-06 19:12:44 +03:00 |
|
Claudio Atzori
|
8c457f1b2c
|
conflicts resolved, merged from beta
|
2022-04-06 10:27:52 +02:00 |
|
Miriam Baglioni
|
e77d104951
|
[OC] added / to workflow path
|
2022-04-05 15:07:11 +02:00 |
|
Sandro La Bruzzo
|
1b11010169
|
minor fix
|
2022-03-29 10:59:14 +02:00 |
|
Claudio Atzori
|
a87c070447
|
conflicts resolved, merged from beta
|
2022-02-24 12:51:31 +01:00 |
|
Claudio Atzori
|
5226d0a100
|
Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta
|
2022-02-18 15:21:07 +01:00 |
|
Claudio Atzori
|
401dd38074
|
code formatting
|
2022-02-18 15:19:15 +01:00 |
|
Sandro La Bruzzo
|
891781ee3f
|
Merge branch 'beta' of code-repo.d4science.org:D-Net/dnet-hadoop into beta
|
2022-02-18 11:11:32 +01:00 |
|
Sandro La Bruzzo
|
d3f03abd51
|
fixed wrong json path
|
2022-02-18 11:11:17 +01:00 |
|
Claudio Atzori
|
89c7313fc5
|
Merge branch 'beta' into hierarchical_orgs_relations
|
2022-02-17 10:30:04 +01:00 |
|
Sandro La Bruzzo
|
3aa2020b24
|
added script to regenerate hostedBy Map following instruction defined on ticket #7539
updated hosted By Map
|
2022-02-15 11:05:27 +01:00 |
|
Miriam Baglioni
|
be64055cfe
|
[OpenCitation] changed the name of destination folders
|
2022-02-14 15:49:44 +01:00 |
|
Miriam Baglioni
|
1490867cc7
|
[OpenCitation] cleaning of the COCI model
|
2022-02-14 14:52:12 +01:00 |
|
Miriam Baglioni
|
5c4043dba8
|
[OpenCitation] refactoring
|
2022-02-08 16:23:05 +01:00 |
|
Miriam Baglioni
|
759ed519f2
|
[OpenCitation] added logic to avoid the genration of self citations relations
|
2022-02-08 16:15:34 +01:00 |
|
Miriam Baglioni
|
b071f8e415
|
[OpenCitation] change to extract in json format each folder just onece
|
2022-02-08 15:37:28 +01:00 |
|
Miriam Baglioni
|
fbc28ee8c3
|
[OpenCitation] change the integration logic to consider dois with commas inside
|
2022-02-07 18:32:08 +01:00 |
|
Miriam Baglioni
|
493caef358
|
[stats-wf]fixed the result_result table related to PR#191
|
2022-02-04 14:51:25 +01:00 |
|
Miriam Baglioni
|
73eba34d42
|
[UnresolvedEntities] Changed the way to merge the unresolved because the new merge removed the dataInfo from the merged result. Added also data info for subjects
|
2022-02-01 08:38:41 +01:00 |
|
Claudio Atzori
|
b37bc277c4
|
reintroduced the hostedby patching to the datacite records
|
2022-01-21 09:15:13 +01:00 |
|
Miriam Baglioni
|
a75fb8c47a
|
[BipFinderInstanceLevel] change pom to align to the dhp-schema release 2.10.24 and refactoring
|
2022-01-12 18:06:26 +01:00 |
|
Miriam Baglioni
|
e7d5a39c03
|
[BipFinderInstanceLevel] added tests in test class
|
2022-01-12 17:25:04 +01:00 |
|
Miriam Baglioni
|
4993666d73
|
[BipFinderInstanceLevel] changed creation of the instance to allow to enrich existing instances with same pid
|
2022-01-12 16:53:47 +01:00 |
|
Sandro La Bruzzo
|
57e2c4b749
|
formatted code
|
2022-01-12 09:40:28 +01:00 |
|
Claudio Atzori
|
dcd282977c
|
pulled from beta
|
2022-01-11 16:59:41 +01:00 |
|
Claudio Atzori
|
4f212652ca
|
scalafmt: code formatting
|
2022-01-11 16:57:48 +01:00 |
|
Miriam Baglioni
|
b7e450070b
|
[SDG-FOS] to import SDG file not considering the header
|
2022-01-07 12:13:26 +01:00 |
|
Miriam Baglioni
|
639190370a
|
mergin with branch beta
|
2022-01-07 11:29:25 +01:00 |
|
Miriam Baglioni
|
adccc2346a
|
[SDG-FOS] to lower case for the doi
|
2022-01-07 11:28:50 +01:00 |
|
Claudio Atzori
|
3bd3653be9
|
OAF-store-graph mdstores: save them in text format
|
2022-01-04 16:39:39 +01:00 |
|
Claudio Atzori
|
3dc48c7ab5
|
OAF-store-graph mdstores: save them in text format
|
2022-01-04 16:39:27 +01:00 |
|
Claudio Atzori
|
f82db765db
|
OAF-store-graph mdstores: save them in text format
|
2022-01-04 16:39:15 +01:00 |
|
Claudio Atzori
|
9458ee7938
|
serialise records in the OAF-store-graph mdstores in json format. Read them again in the graph construction phase using a tolerant parser to support backward compatible changes in the evolution of the schema
|
2022-01-04 16:38:09 +01:00 |
|
Claudio Atzori
|
58f8998e3d
|
OAF-store-graph mdstores: save them in text format
|
2022-01-04 15:02:09 +01:00 |
|
Claudio Atzori
|
174c3037e1
|
OAF-store-graph mdstores: save them in text format
|
2022-01-04 14:40:16 +01:00 |
|
Claudio Atzori
|
045d767013
|
OAF-store-graph mdstores: save them in text format
|
2022-01-04 14:23:01 +01:00 |
|
Claudio Atzori
|
a6977197b3
|
serialise records in the OAF-store-graph mdstores in json format. Read them again in the graph construction phase using a tolerant parser to support backward compatible changes in the evolution of the schema
|
2022-01-03 17:25:26 +01:00 |
|
Miriam Baglioni
|
92fd69e25d
|
[SDG-FOS] alternative way to get input data to avoid OOM error while getting csv
|
2022-01-03 15:23:06 +01:00 |
|
Miriam Baglioni
|
7a1b440413
|
[SDG] logic to create unresolved entities out of SDG input. This changes also some classes related to FOS to reuse the same code. The code under createunresolvedentities create results with the merged update of the the inputs provided (bip at the level of the isntance, fos and sdg for subjects)
|
2021-12-23 13:24:28 +01:00 |
|
Miriam Baglioni
|
2a67ee13ec
|
[SDG] added model class
|
2021-12-23 10:37:52 +01:00 |
|
Miriam Baglioni
|
10579c0dd0
|
[FOS]fixed doi value in test
|
2021-12-22 23:10:16 +01:00 |
|
Miriam Baglioni
|
6116fc5d40
|
[FOS]added logic to include only different subjects. Test refactoring and extention
|
2021-12-22 23:04:22 +01:00 |
|
Miriam Baglioni
|
b81efb6a9d
|
[FOS]changed the mapping between the csv and the model. Changed Test classes and resources
|
2021-12-22 21:40:35 +01:00 |
|
Miriam Baglioni
|
de6c4c8968
|
[FOS]creation of the unresolved entities: remove the split for the doi: no more needed since each row is related to one doi
|
2021-12-22 16:44:44 +01:00 |
|
Miriam Baglioni
|
34ac56565d
|
refactoring
|
2021-12-22 16:28:11 +01:00 |
|
Miriam Baglioni
|
20ef1d657f
|
refactoring
|
2021-12-22 16:26:36 +01:00 |
|
Miriam Baglioni
|
813f856d3f
|
[BipFinder] removing left over parameter in wf
|
2021-12-22 16:11:12 +01:00 |
|
Miriam Baglioni
|
2c126ed014
|
[BipFinder] create unresolved entities with measures at the level of the instance
|
2021-12-22 16:03:41 +01:00 |
|
Miriam Baglioni
|
0807fdb65a
|
[BipFinder] remove not needed resources
|
2021-12-22 15:37:00 +01:00 |
|
Miriam Baglioni
|
b5e11a3a0a
|
[BipFinder] put in common package BipFinder model
|
2021-12-22 15:33:05 +01:00 |
|
Miriam Baglioni
|
c5739c4266
|
[BipFinder] create action set for the measures at the level of the result
|
2021-12-22 15:08:33 +01:00 |
|
Miriam Baglioni
|
e24a7f3496
|
mergin with branch beta
|
2021-12-21 13:57:19 +01:00 |
|
Miriam Baglioni
|
d1ae219cb4
|
Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta
|
2021-12-21 13:55:53 +01:00 |
|
Sandro La Bruzzo
|
3920d68992
|
Fixed workflow generation of delta in datacite
|
2021-12-21 11:41:49 +01:00 |
|
Sandro La Bruzzo
|
b881ee5ef8
|
[scholexplorer]
- implemented generation of scholix of delta update of datacite
|
2021-12-15 11:25:32 +01:00 |
|
Miriam Baglioni
|
22d4b5619b
|
[BipFinder Result] last changes to test and resources files
|
2021-12-14 14:54:13 +01:00 |
|
Miriam Baglioni
|
6fb6236cd4
|
changed the way to produce the AS for bipFinder.
|
2021-12-14 14:51:14 +01:00 |
|
Miriam Baglioni
|
4eb8276493
|
-
|
2021-12-14 11:12:17 +01:00 |
|
Miriam Baglioni
|
b113586207
|
resolved conflicts
|
2021-12-07 10:16:14 +01:00 |
|
Miriam Baglioni
|
d9836f0cf3
|
[OpenCitations] fixed test when executed one after the other
|
2021-12-06 15:27:09 +01:00 |
|
Sandro La Bruzzo
|
7af0bbd0b1
|
[scala-refactor] Module dhp-aggregation:
Moved all scala source into src/main/scala and src/test/scala
|
2021-12-06 11:26:36 +01:00 |
|
Miriam Baglioni
|
f430688ff7
|
Merge branch 'beta' of https://code-repo.d4science.org/D-Net/dnet-hadoop into beta
|
2021-12-03 12:36:08 +01:00 |
|
Sandro La Bruzzo
|
f7011b90d8
|
format code
|
2021-12-03 11:15:09 +01:00 |
|
Miriam Baglioni
|
87eedad898
|
-
|
2021-12-02 13:17:19 +01:00 |
|
Sandro La Bruzzo
|
feea154e89
|
remove working dir after test
|
2021-11-25 11:02:38 +01:00 |
|
Sandro La Bruzzo
|
028a8acad8
|
add test resources
|
2021-11-25 10:54:47 +01:00 |
|
Sandro La Bruzzo
|
2164a2a889
|
Datacite: Code Refactor generated a general SparkApplication Scala where all the spark scala have to inherit
Commented a little the Datacite transformation code
|
2021-11-25 10:54:13 +01:00 |
|
Sandro La Bruzzo
|
a7cf277d98
|
Datacite: Removed HostedBy Patch as described on ticket #7219, Now all the records will have hosted by Unknown Repository
|
2021-11-22 16:03:17 +01:00 |
|
Sandro La Bruzzo
|
fc03c99805
|
fixed javadocs url after deploying site
|
2021-11-19 10:46:33 +01:00 |
|
Claudio Atzori
|
bd9a43cefd
|
Revert to 4094f2bb9a
|
2021-11-19 09:20:43 +01:00 |
|
Claudio Atzori
|
3a4d925386
|
Merge branch 'beta' into hierarchical_orgs_relations
|
2021-11-18 18:07:08 +01:00 |
|
Sandro La Bruzzo
|
2506d7a679
|
Merge branch 'mvn_site_documentation' of code-repo.d4science.org:D-Net/dnet-hadoop into mvn_site_documentation
|
2021-11-17 11:07:24 +01:00 |
|
Sandro La Bruzzo
|
cded363b55
|
code refactor, created and moved scala code on the correct maven folder under src/main/scala and src/test/scala
|
2021-11-17 11:06:35 +01:00 |
|
Miriam Baglioni
|
4094f2bb9a
|
added integration md file
|
2021-11-17 10:04:52 +01:00 |
|
Miriam Baglioni
|
ec8b0219ff
|
[Documentation] Added first page for Integration via unresolved entities generation
|
2021-11-16 17:41:34 +01:00 |
|
Sandro La Bruzzo
|
2d67020c59
|
added dhp-enrichment maven site template
|
2021-11-16 16:01:08 +01:00 |
|
Claudio Atzori
|
bafa2990f3
|
code formatting
|
2021-11-15 17:07:16 +01:00 |
|
Sandro La Bruzzo
|
efa09057db
|
Merge branch 'beta' of code-repo.d4science.org:D-Net/dnet-hadoop into beta
|
2021-11-15 14:32:09 +01:00 |
|
Sandro La Bruzzo
|
48923e46a1
|
added documentation to Pubmed Class and also added mvn site for dhp-aggregations
|
2021-11-15 14:32:01 +01:00 |
|
Miriam Baglioni
|
4ec88c718c
|
merge with beta - resolved conflict in pom
|
2021-11-15 10:52:16 +01:00 |
|
Miriam Baglioni
|
6f1a434e90
|
[Bypass Action Set] Fixed test to consider the new identifier utils
|
2021-11-15 09:59:23 +01:00 |
|
Miriam Baglioni
|
157d33ebf9
|
[Bypass Action Set] Refactoring
|
2021-11-15 09:58:48 +01:00 |
|
Miriam Baglioni
|
92d0e18b55
|
[Bypass Action Set] used constant DOI instead of "doi"
|
2021-11-12 10:56:58 +01:00 |
|
Miriam Baglioni
|
881113743f
|
[Bypass Action Set] refactoring
|
2021-11-12 10:55:50 +01:00 |
|
Miriam Baglioni
|
47ccb53c4f
|
[Bypass Action Set] modification for comment #157 (comment)
|
2021-11-12 10:54:09 +01:00 |
|
Miriam Baglioni
|
716021546e
|
[Bypass Action Set] minor fix
|
2021-11-12 10:18:01 +01:00 |