Miriam Baglioni
|
10579c0dd0
|
[FOS]fixed doi value in test
|
2021-12-22 23:10:16 +01:00 |
Miriam Baglioni
|
6116fc5d40
|
[FOS]added logic to include only different subjects. Test refactoring and extention
|
2021-12-22 23:04:22 +01:00 |
Miriam Baglioni
|
b81efb6a9d
|
[FOS]changed the mapping between the csv and the model. Changed Test classes and resources
|
2021-12-22 21:40:35 +01:00 |
Miriam Baglioni
|
34ac56565d
|
refactoring
|
2021-12-22 16:28:11 +01:00 |
Miriam Baglioni
|
2c126ed014
|
[BipFinder] create unresolved entities with measures at the level of the instance
|
2021-12-22 16:03:41 +01:00 |
Miriam Baglioni
|
0807fdb65a
|
[BipFinder] remove not needed resources
|
2021-12-22 15:37:00 +01:00 |
Miriam Baglioni
|
22d4b5619b
|
[BipFinder Result] last changes to test and resources files
|
2021-12-14 14:54:13 +01:00 |
Miriam Baglioni
|
4eb8276493
|
-
|
2021-12-14 11:12:17 +01:00 |
Miriam Baglioni
|
d9836f0cf3
|
[OpenCitations] fixed test when executed one after the other
|
2021-12-06 15:27:09 +01:00 |
Sandro La Bruzzo
|
7af0bbd0b1
|
[scala-refactor] Module dhp-aggregation:
Moved all scala source into src/main/scala and src/test/scala
|
2021-12-06 11:26:36 +01:00 |
Sandro La Bruzzo
|
f7011b90d8
|
format code
|
2021-12-03 11:15:09 +01:00 |
Sandro La Bruzzo
|
feea154e89
|
remove working dir after test
|
2021-11-25 11:02:38 +01:00 |
Sandro La Bruzzo
|
028a8acad8
|
add test resources
|
2021-11-25 10:54:47 +01:00 |
Sandro La Bruzzo
|
2164a2a889
|
Datacite: Code Refactor generated a general SparkApplication Scala where all the spark scala have to inherit
Commented a little the Datacite transformation code
|
2021-11-25 10:54:13 +01:00 |
Sandro La Bruzzo
|
a7cf277d98
|
Datacite: Removed HostedBy Patch as described on ticket #7219, Now all the records will have hosted by Unknown Repository
|
2021-11-22 16:03:17 +01:00 |
Claudio Atzori
|
3a4d925386
|
Merge branch 'beta' into hierarchical_orgs_relations
|
2021-11-18 18:07:08 +01:00 |
Claudio Atzori
|
bafa2990f3
|
code formatting
|
2021-11-15 17:07:16 +01:00 |
Miriam Baglioni
|
4ec88c718c
|
merge with beta - resolved conflict in pom
|
2021-11-15 10:52:16 +01:00 |
Miriam Baglioni
|
6f1a434e90
|
[Bypass Action Set] Fixed test to consider the new identifier utils
|
2021-11-15 09:59:23 +01:00 |
Miriam Baglioni
|
716021546e
|
[Bypass Action Set] minor fix
|
2021-11-12 10:18:01 +01:00 |
Miriam Baglioni
|
935062edec
|
[Bypass Action Set] creation of unresolved entities
|
2021-11-11 16:11:25 +01:00 |
Claudio Atzori
|
d02caef185
|
Merge branch 'beta' into hierarchical_orgs_relations
|
2021-10-27 15:36:29 +02:00 |
Michele Artini
|
d66e20e7ac
|
added hierarchy rel in ROR actionset
|
2021-10-21 15:51:48 +02:00 |
Sandro La Bruzzo
|
aeeebd573b
|
code refactor renamed datacite package
|
2021-10-20 17:37:42 +02:00 |
Miriam Baglioni
|
1cc09adfaa
|
Opencitations: chenaged the test class to mirror the creation or not of duplicate dois for .refs oc original plus added optional parameter to duplicate the relation
|
2021-10-18 14:11:27 +02:00 |
Sandro La Bruzzo
|
7b15b88d4c
|
renamed wrong package, implemented last aggregation workflow for scholexplorer
|
2021-10-15 15:00:15 +02:00 |
Sandro La Bruzzo
|
5606014b17
|
code refactor see ticket #7065
|
2021-10-12 08:11:53 +02:00 |
Miriam Baglioni
|
5ec69889db
|
OpenCitations: creation of AS from OC
|
2021-09-27 16:02:06 +02:00 |
Alessia Bardi
|
931f430129
|
Merge branch 'beta' into datasource_model_eosc_beta
|
2021-08-23 11:57:21 +02:00 |
Claudio Atzori
|
baed5e3337
|
test classes moved in specific components
|
2021-08-13 12:14:47 +02:00 |
Claudio Atzori
|
3359f73fcf
|
cleanup & best practices
|
2021-08-13 12:00:42 +02:00 |
Miriam Baglioni
|
32fd75691f
|
refactoring
|
2021-08-13 10:15:42 +02:00 |
Miriam Baglioni
|
ed183d878e
|
GetCSV refactoring - modified test classes due to change in the model of projects and programme
|
2021-08-13 09:28:51 +02:00 |
Miriam Baglioni
|
8769dd8eef
|
GetCSV refactoring - refactoring due to movement of classes
|
2021-08-12 18:20:56 +02:00 |
Miriam Baglioni
|
9da74b544a
|
GetCSV refactoring - refactoring due to movement of classes
|
2021-08-12 18:12:15 +02:00 |
Miriam Baglioni
|
804589eb30
|
reverting
|
2021-08-11 17:23:35 +02:00 |
Miriam Baglioni
|
d688749ad9
|
reverting
|
2021-08-11 17:22:28 +02:00 |
Miriam Baglioni
|
524c06e028
|
reverting
|
2021-08-11 17:20:30 +02:00 |
Miriam Baglioni
|
7aa3260729
|
reverting
|
2021-08-11 17:18:45 +02:00 |
Miriam Baglioni
|
55fc500d8d
|
reverting
|
2021-08-11 17:17:48 +02:00 |
Miriam Baglioni
|
8da3a25cf6
|
merging with branch beta
|
2021-08-11 15:55:34 +02:00 |
Claudio Atzori
|
61d811ba53
|
suggestions from intellij
|
2021-08-11 12:18:20 +02:00 |
Claudio Atzori
|
2ee21da43b
|
suggestions from SonarLint
|
2021-08-11 12:13:22 +02:00 |
Miriam Baglioni
|
708d0ade34
|
Merge branch 'beta' into hostedbymap
|
2021-07-28 10:37:22 +02:00 |
Sandro La Bruzzo
|
825d9f0289
|
fixed datacite workflow starting from Importing delta
|
2021-07-27 16:09:46 +02:00 |
Miriam Baglioni
|
74f801b689
|
mergin with branch beta
|
2021-07-27 13:18:31 +02:00 |
Miriam Baglioni
|
eb07f7f40f
|
Hosted By Map
|
2021-07-27 12:27:26 +02:00 |
Sandro La Bruzzo
|
62ae36a3d2
|
fixed NPE
|
2021-07-22 15:41:38 +02:00 |
Sandro La Bruzzo
|
bbe8193930
|
merged stable ids
|
2021-07-12 17:00:43 +02:00 |
Andreas Czerniak
|
ebf3f47a02
|
from&until more OAI2.0 compl., adding tfs
|
2021-07-07 09:29:49 +02:00 |
Sandro La Bruzzo
|
507e42102a
|
added pdb to oaf class
|
2021-06-21 09:36:40 +02:00 |
Sandro La Bruzzo
|
cc0f2b11fb
|
Implemented mapping from pubmed baseline to OAF
|
2021-06-16 14:56:24 +02:00 |
Sandro La Bruzzo
|
aeb8132627
|
Merged branch stable_ids
|
2021-06-14 10:07:29 +02:00 |
Claudio Atzori
|
a900bfb874
|
delegating the date parsing to https://github.com/sisyphsu/dateparser
|
2021-06-11 16:53:01 +02:00 |
Sandro La Bruzzo
|
5b724d9972
|
added relations to datacite mapping
|
2021-06-04 10:14:22 +02:00 |
Claudio Atzori
|
d512062b58
|
integrating pull #109, H2020Classification
|
2021-05-27 12:22:47 +02:00 |
Miriam Baglioni
|
abd88f663d
|
changed test resource to mirror change in the input file
|
2021-05-21 15:20:47 +02:00 |
Miriam Baglioni
|
53b9d87fec
|
new prepareProgramme according to the new file
|
2021-05-21 11:49:31 +02:00 |
Miriam Baglioni
|
052c837843
|
-
|
2021-05-20 15:54:44 +02:00 |
Claudio Atzori
|
b695932ae4
|
integrated pull#108
|
2021-05-20 15:34:04 +02:00 |
Claudio Atzori
|
239d0f0a9a
|
ROR actionset import workflow backported from branch stable_ids
|
2021-05-18 16:12:11 +02:00 |
Claudio Atzori
|
23b8883ab1
|
applied intellij code cleanup
|
2021-05-14 10:58:12 +02:00 |
Sandro La Bruzzo
|
6424cd9062
|
Added passing of the following parameters:
-varDataSourceId
-varOfficialName
in Each transformation Rule
|
2021-05-11 15:17:38 +02:00 |
Sandro La Bruzzo
|
073dcea2aa
|
Added passing of the following parameters:
-varDataSourceId
-varOfficialName
in Each transformation Rule
|
2021-05-11 15:05:58 +02:00 |
Claudio Atzori
|
3797543600
|
MDStoreManager model classes moved in dhp-schemas
|
2021-05-10 14:32:05 +02:00 |
Michele Artini
|
a278d67175
|
parse input file
|
2021-04-29 11:34:47 +02:00 |
Michele Artini
|
f77ba34126
|
pid types
|
2021-04-29 09:50:05 +02:00 |
Michele Artini
|
7c5cd86927
|
annotations and tests
|
2021-04-29 09:29:19 +02:00 |
Claudio Atzori
|
fa42026590
|
fixed PersonCleaner extension functions
|
2021-04-27 10:10:06 +02:00 |
Sandro La Bruzzo
|
dbe0d0378e
|
resolved ticket #6377
|
2021-04-20 09:44:44 +02:00 |
Sandro La Bruzzo
|
cdfe01bbae
|
improved parallelization on transformation job
|
2021-04-19 15:14:52 +02:00 |
Claudio Atzori
|
3125cef545
|
code formatting
|
2021-04-14 09:11:54 +02:00 |
Andreas Czerniak
|
3b694074ff
|
add xslt, personname cleaner
|
2021-04-13 07:04:27 +02:00 |
Sandro La Bruzzo
|
616d2ecce2
|
splitted workflow collecting datacite into two workflows.
Released on beta
|
2021-03-31 15:45:58 +02:00 |
Claudio Atzori
|
8db248aa13
|
avoiding error on jenkins compilations: java.net.BindException: Cannot assign requested address: Service 'sparkDriver' failed after 16 retries (on a random free port)!
|
2021-03-23 09:56:34 +01:00 |
Claudio Atzori
|
acbe3119a4
|
RestCollectorPlugin imported from dne45
|
2021-03-08 09:44:09 +01:00 |
Claudio Atzori
|
fa7930d2e2
|
merging contributions from PR#97
|
2021-03-05 15:45:28 +01:00 |
Claudio Atzori
|
36f750cd1d
|
removed unused classes
|
2021-03-03 10:22:29 +01:00 |
Claudio Atzori
|
e76c4f62c1
|
MetadataRecord moved in dhp-schemas
|
2021-02-26 10:58:48 +01:00 |
Claudio Atzori
|
271e88537b
|
code formatting
|
2021-02-25 12:28:56 +01:00 |
Claudio Atzori
|
9c899f4433
|
cleanup on transformation functions and the relative tests
|
2021-02-24 15:07:59 +01:00 |
Claudio Atzori
|
1abe6d1ad7
|
WIP: collectorWorker error reporting, added report messages
|
2021-02-15 15:08:59 +01:00 |
Claudio Atzori
|
523a6bfa97
|
Merge pull request 'first commit to the correct branch' (#94) from andreas.czerniak/BrAggr_dnet-hadoop:hadoop_aggregator into hadoop_aggregator
Looks good to me, thanks Andreas!
|
2021-02-15 12:15:31 +01:00 |
Sandro La Bruzzo
|
7edcc87ed4
|
changed xslt behaviour on failure
|
2021-02-12 17:27:08 +01:00 |
Sandro La Bruzzo
|
6a37c7f175
|
merge fixed
|
2021-02-12 16:38:47 +01:00 |
Sandro La Bruzzo
|
b3f5c2351d
|
Merge branch 'hadoop_aggregator' of code-repo.d4science.org:D-Net/dnet-hadoop into hadoop_aggregator
Conflicts:
dhp-workflows/dhp-aggregation/src/test/java/eu/dnetlib/dhp/transformation/TransformationJobTest.java
|
2021-02-12 16:37:14 +01:00 |
Sandro La Bruzzo
|
f216277219
|
Implemented cleaning date
|
2021-02-12 16:34:52 +01:00 |
Andreas Czerniak
|
5a9017cf18
|
clone, min. changes, test, run
|
2021-02-12 14:32:36 +01:00 |
Claudio Atzori
|
aa55dedb8a
|
Merge branch 'hadoop_aggregator' of https://code-repo.d4science.org/D-Net/dnet-hadoop into hadoop_aggregator
|
2021-02-12 12:31:05 +01:00 |
Claudio Atzori
|
29c6f7e255
|
classes related to the collection workflow moved into common package; implemented MongoDB collection plugins
|
2021-02-12 12:31:02 +01:00 |
Sandro La Bruzzo
|
17e6f1934e
|
fixed NPE on cleaner
|
2021-02-12 11:48:11 +01:00 |
Claudio Atzori
|
50add4c61b
|
added requestDelay to HttpConnector2 configuration; Aggregation workflow constants moved in dhp-common
|
2021-02-08 12:19:38 +01:00 |
Claudio Atzori
|
40df0f987d
|
better logging, WIP: collectorWorker error reporting; common functions moved in DHPUtils
|
2021-02-06 20:12:00 +01:00 |
Claudio Atzori
|
a8a758925e
|
better logging, WIP: collectorWorker error reporting
|
2021-02-05 19:18:05 +01:00 |
Claudio Atzori
|
730973679a
|
Merge branch 'hadoop_aggregator' of https://code-repo.d4science.org/D-Net/dnet-hadoop into hadoop_aggregator
|
2021-02-04 17:25:00 +01:00 |
Claudio Atzori
|
deb85706db
|
imported HttpConnector from https://svn.driver.research-infrastructures.eu/driver/dnet45/modules/dnet-modular-collector-service/trunk/src/main/java/eu/dnetlib/data/collector/plugins/HttpConnector.java as HttpConnector2
|
2021-02-04 17:24:52 +01:00 |
Sandro La Bruzzo
|
4dae5e605d
|
implemented messaging btween collection worker and Dnet
|
2021-02-04 15:51:15 +01:00 |
Claudio Atzori
|
40764cf626
|
better logging, WIP: collectorWorker error reporting
|
2021-02-04 14:06:02 +01:00 |
Sandro La Bruzzo
|
69c253710b
|
fixed test
|
2021-02-04 10:30:49 +01:00 |
Claudio Atzori
|
0e8a4f9f1a
|
better logging, WIP: collectorWorker error reporting
|
2021-02-03 12:33:41 +01:00 |
Claudio Atzori
|
bb89b99b24
|
code formatting
|
2021-02-02 12:34:14 +01:00 |
Claudio Atzori
|
75807ea5ae
|
factored out constants
|
2021-02-02 12:28:21 +01:00 |
Sandro La Bruzzo
|
0634674add
|
implemented transformation test
|
2021-02-02 12:12:14 +01:00 |
Claudio Atzori
|
8eaa1fd4b4
|
WIP: metadata collection in INCREMENTAL mode and relative test
|
2021-02-01 19:29:10 +01:00 |
Sandro La Bruzzo
|
6ff234d81b
|
Implemented a first prototype of incremental harvesting and trasformation using readlock
|
2021-02-01 13:56:05 +01:00 |
Sandro La Bruzzo
|
8ee82576c6
|
Collection on Refresh WORKS!!!
|
2021-01-29 17:02:46 +01:00 |
Sandro La Bruzzo
|
0276180039
|
WIP mdstore
transaction implemented on hadoop side
|
2021-01-29 16:42:41 +01:00 |
Sandro La Bruzzo
|
98b9498b57
|
Removed old messaging system not quite used from collection and Transformation workflow
code refactor
|
2021-01-28 09:51:17 +01:00 |
Sandro La Bruzzo
|
184e7b3856
|
Implemented new Transformation using spark
|
2021-01-27 15:43:08 +01:00 |
Sandro La Bruzzo
|
ffb092b8d3
|
removed duplicate code HttpConnector.java
|
2021-01-25 15:05:37 +01:00 |
Claudio Atzori
|
41500669e2
|
[BIP! Scores integration] merged missing classes from bipFinder branch
|
2021-01-11 14:39:47 +01:00 |
Claudio Atzori
|
03319d3bd9
|
Revert "Merge pull request 'Creation of the action set to include the bipFinder! score' (#62) from miriam.baglioni/dnet-hadoop:bipFinder into master"
This reverts commit add7e1693b , reversing
changes made to f9a8fd8bbd .
|
2020-12-17 12:23:58 +01:00 |
Miriam Baglioni
|
5b3ed70808
|
refactoring
|
2020-12-01 14:31:34 +01:00 |
Miriam Baglioni
|
0051ebede5
|
extending test
|
2020-12-01 12:43:03 +01:00 |
Miriam Baglioni
|
719da15f04
|
added test resources
|
2020-12-01 12:42:30 +01:00 |
Miriam Baglioni
|
db36e11912
|
classes test classes and resources for production of the actionset to include bipFinder score in results
|
2020-11-30 20:14:23 +01:00 |
Sandro La Bruzzo
|
66efb39634
|
implemented merge scholix
|
2020-11-04 09:04:01 +01:00 |
Miriam Baglioni
|
4905739be6
|
changed resource file to mirror change in business logic
|
2020-10-30 17:02:57 +01:00 |
Miriam Baglioni
|
b40360ebfb
|
changed the code to mirror the changed decision in the classification level and prodramme description labels
|
2020-10-30 17:02:30 +01:00 |
Miriam Baglioni
|
696409fb9f
|
disabled tests because needing remote resource
|
2020-10-30 17:01:48 +01:00 |
Miriam Baglioni
|
a2ce527fae
|
changed to match the requirements for short titles in level and long titles in classification
|
2020-10-20 17:03:25 +02:00 |
Miriam Baglioni
|
61946b4092
|
refactoring
|
2020-10-01 16:22:48 +02:00 |
Miriam Baglioni
|
7e6d35e56c
|
added the link to the excel file related to topic
|
2020-10-01 15:53:31 +02:00 |
Miriam Baglioni
|
632351c0da
|
modified test resources to mirror the changed in the code
|
2020-10-01 15:43:02 +02:00 |
Miriam Baglioni
|
ebc1c5513f
|
modified test resources to mirror the changed in the code
|
2020-10-01 15:42:29 +02:00 |
Miriam Baglioni
|
83ea746163
|
added check to the test
|
2020-10-01 15:40:28 +02:00 |
Miriam Baglioni
|
a46179f61c
|
refactoring
|
2020-10-01 11:22:01 +02:00 |
Miriam Baglioni
|
c107f193c9
|
refactoring
|
2020-10-01 11:16:22 +02:00 |
Miriam Baglioni
|
706a80a29a
|
added test to check that separator '-' (not hyphen) will be recognized
|
2020-10-01 10:38:31 +02:00 |
Miriam Baglioni
|
3dca586b3b
|
refactoring
|
2020-10-01 10:34:48 +02:00 |
Miriam Baglioni
|
f4739a371a
|
code to get the information related to the topic association between code and description.
|
2020-09-28 12:02:48 +02:00 |
Miriam Baglioni
|
12c2dfc268
|
modified the resource to consider the information added to the model
|
2020-09-25 14:17:23 +02:00 |
Miriam Baglioni
|
969fa8d96e
|
fixed issue and changed the transformation of the programme file to consider the new model
|
2020-09-25 13:32:34 +02:00 |
Miriam Baglioni
|
e917281822
|
-
|
2020-09-24 15:24:05 +02:00 |
Miriam Baglioni
|
1d84cf19a6
|
added new line to resource file
|
2020-09-23 17:32:22 +02:00 |
Miriam Baglioni
|
f0c476b6c9
|
modification to the test classes to consider h2020classification
|
2020-09-23 17:31:49 +02:00 |
Miriam Baglioni
|
1069cf243a
|
modification to the schema to consider the H2020classification of the programme. The filed Programme has been moved inside the H2020classification that is now associated to the Project. Programme is no more associated directly to the Project but via H2020CLassification
|
2020-09-22 14:38:00 +02:00 |
Claudio Atzori
|
306669209f
|
code formatting
|
2020-06-16 16:54:44 +02:00 |
Claudio Atzori
|
603b1bd0bb
|
Merge branch 'master' into dhp_oaf_model
|
2020-06-16 15:43:59 +02:00 |
Claudio Atzori
|
a2fdf85ba1
|
WIP: graph cleaner implementation
|
2020-06-09 19:52:53 +02:00 |
Miriam Baglioni
|
dfa4997a4f
|
removed commented code
|
2020-05-29 10:45:18 +02:00 |
Miriam Baglioni
|
8b6e886fb6
|
added new resource for testing
|
2020-05-28 23:54:31 +02:00 |
Miriam Baglioni
|
6989fb9c8a
|
changed the project test according to the newly introduced join with the db project codes
|
2020-05-28 23:53:24 +02:00 |
Miriam Baglioni
|
01f7876595
|
fix issue with flatMap - the return type must not be null
|
2020-05-28 23:50:32 +02:00 |
Miriam Baglioni
|
35b7279147
|
changed test because data are saved as SequenceFile now, and because of the group by the umber of produced update decrease
|
2020-05-28 10:26:12 +02:00 |
Miriam Baglioni
|
ac8025f469
|
-
|
2020-05-22 15:29:41 +02:00 |
Miriam Baglioni
|
50ad83b97f
|
-
|
2020-05-22 15:27:19 +02:00 |
Miriam Baglioni
|
055eec5a77
|
added resource for prepare project test
|
2020-05-20 13:54:10 +02:00 |
Miriam Baglioni
|
9079bc1f61
|
-
|
2020-05-20 13:53:32 +02:00 |
Miriam Baglioni
|
67ba4fde57
|
added test for prepare projects step
|
2020-05-20 13:53:08 +02:00 |