 Enrico Ottonello
|
f8e6e04e82
|
added charset on payload creation for nara publishing
|
2022-01-31 15:25:38 +01:00 |
|
Enrico Ottonello
|
98cccc4442
|
removed GZIPCompression on request, currently it is not needed
|
2021-12-17 11:44:16 +01:00 |
|
Enrico Ottonello
|
a90f02487a
|
record to publish now are sent to spring module inside body request, because of htmlsimple harvesting can produce large record (>10MB)
|
2021-12-17 11:39:37 +01:00 |
 Alessia Bardi
|
0d0ebc4d77
|
testing DIME
|
2021-10-25 15:24:23 +02:00 |
|
Enrico Ottonello
|
788fc3b734
|
added lowercase function for derivedSubject.prefLabe, nativeSubject.prefLabel
|
2021-10-19 16:03:38 +02:00 |
|
Enrico Ottonello
|
e9430ab460
|
changed title structure from field to object for multiple language handling
|
2021-10-18 18:06:40 +02:00 |
|
Enrico Ottonello
|
3510d7d1be
|
Merge branch 'new_es_mapping'
# Conflicts:
# dnet-ariadneplus-graphdb-publisher/src/main/resources/eu/dnetlib/ariadneplus/sparql/read_collection_data_template.sparql
# dnet-ariadneplus-graphdb-publisher/src/main/resources/eu/dnetlib/ariadneplus/sparql/read_record_data_template.sparql
# dnet-ariadneplus-graphdb-publisher/test/java/eu/dnetlib/ariadneplus/GraphDbReaderAndESIndexTest.java
|
2021-09-08 15:56:12 +02:00 |
|
Enrico Ottonello
|
03da34a897
|
added dynamic property set to false to avoid es mapping modification indexinga record; previous index has been deleted
|
2021-06-15 23:38:02 +02:00 |
|
Alessia Bardi
|
fc71bc24ed
|
new generator policy
|
2021-06-04 10:20:46 +02:00 |
|
Enrico Ottonello
|
06f20f7a8d
|
shell command to create the new elastic search index with mapping on staging
|
2021-05-31 16:59:56 +02:00 |
|
Enrico Ottonello
|
5a68ab07b9
|
mapping loaded only once now
|
2021-01-29 17:16:48 +01:00 |
|
Enrico Ottonello
|
290259fbc2
|
new collector plugin (xmlsFolder) based on ads one, adding a new parameter which represents the list of namespaces that has to be added to the generated xml record during collection step
|
2021-01-15 13:33:23 +01:00 |
|
Enrico Ottonello
|
35b81aeee4
|
Merge branch 'master' of https://code-repo.d4science.org/D-Net/AriadnePlus
|
2021-01-09 00:53:43 +01:00 |
|
Enrico Ottonello
|
a14c66f36e
|
added default values for has_temporal_coverage and has_spatial_coverage
|
2021-01-09 00:53:33 +01:00 |
 Miriam Baglioni
|
472ee2754c
|
Aggiornare 'dnet-ariadneplus/src/main/resources/es/port.py'
|
2021-01-08 16:52:26 +01:00 |
|
Miriam Baglioni
|
060f7f8673
|
Aggiornare 'dnet-ariadneplus/src/main/resources/es/port.py'
|
2021-01-08 16:51:44 +01:00 |
|
Miriam Baglioni
|
538be9e035
|
Aggiungi 'dnet-ariadneplus/src/main/resources/es/port.py'
|
2021-01-08 16:51:07 +01:00 |
|
Enrico Ottonello
|
b8d0e3f741
|
enrichment queries set is now splitted on workflow node side, because of timeout error on very large collection
|
2021-01-07 23:40:47 +01:00 |
|
Enrico Ottonello
|
4673c143ce
|
curl command to create catalog index with mapping for elastic search on production
|
2021-01-07 11:48:00 +01:00 |
|
Enrico Ottonello
|
1ca29a2ebf
|
alignment with current elastic-test instance mapping
|
2021-01-07 11:35:33 +01:00 |
|
Enrico Ottonello
|
a05d8e97da
|
replicated with a new id for having different execution dates with aggregation_to_graphdb_wf.xml
|
2020-12-18 17:39:15 +01:00 |
|
Enrico Ottonello
|
3e68138b81
|
new wf to publish a collection on staging graphdb and then on public one
|
2020-12-18 16:20:38 +01:00 |
|
Enrico Ottonello
|
dc6bd40d1e
|
let isStart true only for collection node, avoiding parallelism on multiple nodes
|
2020-12-18 16:19:43 +01:00 |
|
Alessia Bardi
|
5f695ee538
|
fixed worng copy&paste, probably from an st file...again
|
2020-12-17 20:15:51 +01:00 |
|
Alessia Bardi
|
0e30eccbae
|
fixed worng copy&paste, probably from an st file
|
2020-12-17 17:56:13 +01:00 |
|
Alessia Bardi
|
6c150be5b2
|
added missing param
|
2020-12-17 13:53:26 +01:00 |
|
Alessia Bardi
|
e5eefef411
|
fixed param name
|
2020-12-16 16:09:40 +01:00 |
|
Alessia Bardi
|
fa3592b1b3
|
added missing wf node
|
2020-12-15 17:06:52 +01:00 |
|
Alessia Bardi
|
3e28f8fd57
|
increasing mid version number
|
2020-12-14 18:57:36 +01:00 |
|
Alessia Bardi
|
4edfc4af05
|
new_agg_wf.xml annotated with comments to guide the (manual) migration of old workflows to the new one
|
2020-12-14 18:56:49 +01:00 |
|
Alessia Bardi
|
687d5ab9ec
|
only one start node
|
2020-12-14 18:27:23 +01:00 |
|
Alessia Bardi
|
be64ce5dcf
|
changed workflow name ad description
|
2020-12-14 18:22:13 +01:00 |
|
Alessia Bardi
|
8a9782cb1e
|
re-organising workflows and templates for staging/public instances of graphdb and elasticsearch
|
2020-12-14 17:49:53 +01:00 |
|
Enrico Ottonello
|
5d29077a43
|
added counter on wf progress information
|
2020-10-20 19:34:40 +02:00 |
|
Enrico Ottonello
|
8073ac7131
|
the node retrieves resources identifier list for collection and record, then loops on the 2 identifiers lists and request indexing to rest module
|
2020-10-20 17:43:27 +02:00 |
|
Enrico Ottonello
|
1398c2495e
|
test for new indexing methods on rest module
|
2020-10-20 16:42:44 +02:00 |
|
Enrico Ottonello
|
0e96774895
|
changed http request method for indexing
|
2020-10-15 17:09:35 +02:00 |
|
Enrico Ottonello
|
25de647504
|
set KeepAlive true on used socket; replaced SimpleJobNode with AsyncJobNode
|
2020-10-13 15:34:43 +02:00 |
|
Enrico Ottonello
|
2e732a3b5b
|
sparql insert for ads 1;replaced AsynchJobNode with SimpleJobNode to 404 problem
|
2020-10-10 18:05:02 +02:00 |
|
Enrico Ottonello
|
8b3f8896a5
|
removed progress bar handler attached to resultset client (not used here)
|
2020-10-10 16:53:37 +02:00 |
|
Enrico Ottonello
|
db2ad3f97e
|
multithreads http client not needed on indexjobnode, maybe avoid 404 response after a few minutes, added log on each indexing operation
|
2020-08-12 15:47:55 +02:00 |
|
Enrico Ottonello
|
5c3ef5f25b
|
retrieved body from server response
|
2020-07-24 18:32:28 +02:00 |
|
Enrico Ottonello
|
e189283059
|
retrieved report on index results
|
2020-07-24 17:59:23 +02:00 |
|
Alessia Bardi
|
74d0f440e3
|
Merge branch 'master' of https://code-repo.d4science.org/D-Net/AriadnePlus
|
2020-06-23 10:32:11 +02:00 |
|
Alessia Bardi
|
e5d1f845a3
|
updated policy
|
2020-06-23 10:32:04 +02:00 |
|
Enrico Ottonello
|
db9b70feb0
|
new node and workflow to index on ES
|
2020-06-16 02:36:16 +02:00 |
|
Alessia Bardi
|
4d68ecad58
|
the job fails if the endpoint returns client errors 4xx
|
2020-06-05 16:19:54 +02:00 |
|
Alessia Bardi
|
30410e8cca
|
need to enclose policy in CDATA, otherwise we get an error from the X3ML engine 'only whitespace content allowed before start tag and not C'
|
2020-06-05 16:02:31 +02:00 |
|
Alessia Bardi
|
fa3707a56f
|
updated policy
|
2020-06-05 15:47:25 +02:00 |
|
Alessia Bardi
|
e5cfbd01af
|
ADS plugin now able to process one single remote file
|
2020-06-05 15:20:21 +02:00 |