Claudio Atzori
|
e7eba9f7e7
|
WIP: transformation workflow error reporting; cleanup
|
2021-02-17 16:54:08 +01:00 |
Claudio Atzori
|
58467aaf1e
|
WIP: transformation workflow error reporting
|
2021-02-17 16:14:41 +01:00 |
Claudio Atzori
|
cc88701f29
|
retry for any Socket exception
|
2021-02-17 16:13:54 +01:00 |
Antonis Lempesis
|
a445c1ac3d
|
fixed variable names in monitor script
|
2021-02-17 16:45:09 +02:00 |
Antonis Lempesis
|
00d516360f
|
added missing ;
|
2021-02-17 16:41:10 +02:00 |
Claudio Atzori
|
545f8f3e48
|
using jackson objectmapper instead of GSon to serialise the aggregation report
|
2021-02-17 12:15:00 +01:00 |
Claudio Atzori
|
b592d78bb4
|
WIP: collectorWorker error reporting, generalised reported implementation
|
2021-02-17 10:28:01 +01:00 |
Antonis Lempesis
|
cd1b794409
|
added the monitor db wf
|
2021-02-17 02:11:55 +02:00 |
Claudio Atzori
|
cf27905a71
|
WIP: collectorWorker error reporting, added report messages
|
2021-02-16 16:53:14 +01:00 |
Alessia Bardi
|
bf2830b981
|
Merge pull request 'manage merging of Relatation validation attributes' (#95) from merge_rel_validation into master
|
2021-02-16 12:14:27 +01:00 |
Claudio Atzori
|
6f9864c564
|
manage merging of Relatation validation attributes
|
2021-02-16 11:53:44 +01:00 |
Alessia Bardi
|
32e81c2d89
|
non validated rel has null value in validated field
|
2021-02-16 11:01:42 +01:00 |
Claudio Atzori
|
58288a95b8
|
WIP: collectorWorker error reporting, added report messages
|
2021-02-15 15:28:53 +01:00 |
Claudio Atzori
|
1abe6d1ad7
|
WIP: collectorWorker error reporting, added report messages
|
2021-02-15 15:08:59 +01:00 |
Claudio Atzori
|
523a6bfa97
|
Merge pull request 'first commit to the correct branch' (#94) from andreas.czerniak/BrAggr_dnet-hadoop:hadoop_aggregator into hadoop_aggregator
Looks good to me, thanks Andreas!
|
2021-02-15 12:15:31 +01:00 |
Antonis Lempesis
|
1c029b9fc0
|
fixed formatting
|
2021-02-14 03:14:24 +02:00 |
Antonis Lempesis
|
2c4dcc90ba
|
analyzing tables to produce stats
|
2021-02-14 02:54:55 +02:00 |
Sandro La Bruzzo
|
7edcc87ed4
|
changed xslt behaviour on failure
|
2021-02-12 17:27:08 +01:00 |
Sandro La Bruzzo
|
6a37c7f175
|
merge fixed
|
2021-02-12 16:38:47 +01:00 |
Sandro La Bruzzo
|
b3f5c2351d
|
Merge branch 'hadoop_aggregator' of code-repo.d4science.org:D-Net/dnet-hadoop into hadoop_aggregator
Conflicts:
dhp-workflows/dhp-aggregation/src/test/java/eu/dnetlib/dhp/transformation/TransformationJobTest.java
|
2021-02-12 16:37:14 +01:00 |
Sandro La Bruzzo
|
f216277219
|
Implemented cleaning date
|
2021-02-12 16:34:52 +01:00 |
Andreas Czerniak
|
5a9017cf18
|
clone, min. changes, test, run
|
2021-02-12 14:32:36 +01:00 |
Claudio Atzori
|
aa55dedb8a
|
Merge branch 'hadoop_aggregator' of https://code-repo.d4science.org/D-Net/dnet-hadoop into hadoop_aggregator
|
2021-02-12 12:31:05 +01:00 |
Claudio Atzori
|
29c6f7e255
|
classes related to the collection workflow moved into common package; implemented MongoDB collection plugins
|
2021-02-12 12:31:02 +01:00 |
Sandro La Bruzzo
|
17e6f1934e
|
fixed NPE on cleaner
|
2021-02-12 11:48:11 +01:00 |
Sandro La Bruzzo
|
ebcc3ec14f
|
updated wrong datacite identifier in trasformation
|
2021-02-11 16:25:51 +01:00 |
Michele Artini
|
83d815d0bc
|
only stats
|
2021-02-11 10:57:23 +01:00 |
Michele Artini
|
8c836bf930
|
Merge branch 'master' of code-repo.d4science.org:D-Net/dnet-hadoop
|
2021-02-11 10:54:41 +01:00 |
Michele Artini
|
8c1600398a
|
added resumeFrom parameter
|
2021-02-11 10:54:16 +01:00 |
Claudio Atzori
|
3f8f78cbfb
|
Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop
|
2021-02-11 09:36:10 +01:00 |
Claudio Atzori
|
b34b5a39ca
|
index field authoridtypevalue mixes up different author id-type value pairs, dropped in favour of orcidtypevalue
|
2021-02-11 09:36:04 +01:00 |
Michele Artini
|
7249cceb53
|
switch of 2 nodes
|
2021-02-11 09:27:08 +01:00 |
Claudio Atzori
|
73393d3c4d
|
Merge pull request 'validatedLinksToProjects' (#93) from validatedLinksToProjects into master
LGTM
|
2021-02-10 12:32:35 +01:00 |
Alessia Bardi
|
986dd969d3
|
use the proper import for Lists
|
2021-02-10 12:03:54 +01:00 |
miconis
|
4b2124a18e
|
implementation of the openorgs wfs, implementation of the raw_all wf to migrate openorgs db entities
|
2021-02-10 11:51:50 +01:00 |
Alessia Bardi
|
c4d1feca74
|
mapper test with validated link to project
|
2021-02-10 11:22:54 +01:00 |
Alessia Bardi
|
09fc7e2f78
|
serialization of validated flag on relationships
|
2021-02-10 11:22:09 +01:00 |
Enrico Ottonello
|
ee4ba7298b
|
fix last update read/write from file on hdfs
|
2021-02-09 23:24:57 +01:00 |
Claudio Atzori
|
bc458d1b54
|
Merge branch 'master' of https://code-repo.d4science.org/D-Net/dnet-hadoop
|
2021-02-09 16:27:30 +01:00 |
Claudio Atzori
|
82e6c50f3f
|
updated solr fields (authoridtypevalue, resultsubject, resultresourcetypename)
|
2021-02-09 16:27:04 +01:00 |
Claudio Atzori
|
62bd3c53ee
|
Merge branch 'master' into provision_indexing
|
2021-02-09 15:46:26 +01:00 |
Claudio Atzori
|
bae029f828
|
collection_java_xmx allows to declare the heap size allocated for the java actions involved in the metadata collectionw workflow
|
2021-02-08 18:07:23 +01:00 |
Claudio Atzori
|
bebc54d5bf
|
seq file storing native records is now compressed
|
2021-02-08 18:06:25 +01:00 |
Claudio Atzori
|
50add4c61b
|
added requestDelay to HttpConnector2 configuration; Aggregation workflow constants moved in dhp-common
|
2021-02-08 12:19:38 +01:00 |
Miriam Baglioni
|
2f5e6647c6
|
merge upstream
|
2021-02-08 10:33:11 +01:00 |
Claudio Atzori
|
40df0f987d
|
better logging, WIP: collectorWorker error reporting; common functions moved in DHPUtils
|
2021-02-06 20:12:00 +01:00 |
Claudio Atzori
|
a8a758925e
|
better logging, WIP: collectorWorker error reporting
|
2021-02-05 19:18:05 +01:00 |
Michele Artini
|
2ee0c3e47e
|
http entity as json string
|
2021-02-05 09:45:39 +01:00 |
Claudio Atzori
|
730973679a
|
Merge branch 'hadoop_aggregator' of https://code-repo.d4science.org/D-Net/dnet-hadoop into hadoop_aggregator
|
2021-02-04 17:25:00 +01:00 |
Claudio Atzori
|
deb85706db
|
imported HttpConnector from https://svn.driver.research-infrastructures.eu/driver/dnet45/modules/dnet-modular-collector-service/trunk/src/main/java/eu/dnetlib/data/collector/plugins/HttpConnector.java as HttpConnector2
|
2021-02-04 17:24:52 +01:00 |