 Alessia Bardi
|
83ecd94c6f
|
uncommented logs and set them in debug mode
|
2020-10-12 18:15:15 +02:00 |
 Enrico Ottonello
|
2aafbc8506
|
fix loop condition
|
2020-10-10 01:03:41 +02:00 |
|
Enrico Ottonello
|
fe614ce2ed
|
added retry loop on HTTPQueryEvaluationException (Heap space almost full)
|
2020-10-10 00:55:26 +02:00 |
|
Enrico Ottonello
|
142a296166
|
changed exception type, if collection indexing failes
|
2020-09-24 12:53:08 +02:00 |
|
Enrico Ottonello
|
814c908663
|
collection is now indexed before records, if there is an error on collection indexing, wf failed immediately; enriched query for 397 collection
|
2020-09-24 12:35:59 +02:00 |
|
Alessia Bardi
|
12c80713d7
|
indexing result per record logged in debug mode instead of info
|
2020-09-07 15:45:26 +02:00 |
|
Alessia Bardi
|
832b95f064
|
parse must succeed if the date is already in the expected format
|
2020-09-07 15:37:40 +02:00 |
|
Alessia Bardi
|
19ebb63d88
|
fixed log
|
2020-09-07 15:00:09 +02:00 |
|
Alessia Bardi
|
f1e3de1b07
|
avoid null pointer exception
|
2020-09-07 14:59:54 +02:00 |
|
Alessia Bardi
|
4fb5bb2941
|
it is legal to have no individual resources or no collections: let's index what we have
|
2020-09-04 16:57:38 +02:00 |
|
Enrico Ottonello
|
db2ad3f97e
|
multithreads http client not needed on indexjobnode, maybe avoid 404 response after a few minutes, added log on each indexing operation
|
2020-08-12 15:47:55 +02:00 |
|
Alessia Bardi
|
8329f6e9dc
|
useless if removed
|
2020-08-07 16:40:21 +02:00 |
|
Alessia Bardi
|
ca4ad7ea3b
|
Need to split the multiple inserts and perform separate transactions in GraphDB Free edition. See https://stackoverflow.com/questions/54428161/graphdb-read-check-and-update-in-a-transaction
|
2020-08-07 14:54:15 +02:00 |
|
Alessia Bardi
|
69ca7eaade
|
removed redundand lines of code
|
2020-08-07 12:44:36 +02:00 |
|
Enrico Ottonello
|
220c6be6db
|
report returned to wf node
|
2020-07-24 13:15:27 +02:00 |
|
Enrico Ottonello
|
214502cc11
|
removed test publisher
|
2020-07-24 12:59:59 +02:00 |
|
Enrico Ottonello
|
f61f794c25
|
removed a few logs
|
2020-07-24 12:34:06 +02:00 |
|
Enrico Ottonello
|
11e00c806d
|
reset json parser status on each record
|
2020-07-24 01:48:17 +02:00 |
|
Enrico Ottonello
|
b29b4e1fd7
|
added log on json parser
|
2020-07-24 01:21:25 +02:00 |
|
Enrico Ottonello
|
eb61aebcd5
|
added other check on parsing json from rdf statements
|
2020-07-24 00:32:49 +02:00 |
|
Enrico Ottonello
|
5eaa655f01
|
added other check on es return code
|
2020-07-23 23:12:07 +02:00 |
|
Enrico Ottonello
|
173b0c289e
|
to avoid overload on graphdb heap it is better to slowing down query execution
|
2020-07-22 23:47:08 +02:00 |
|
Enrico Ottonello
|
5bb65a8854
|
added response code from es; added sparql insert for copying ariadne_subject value into collection for ads; fix sparql insert for copying collection data into records, collection id condition is needed
|
2020-07-22 23:13:47 +02:00 |
|
Enrico Ottonello
|
766f8199f2
|
removed empty values/structures from record/collection to es
|
2020-07-17 17:42:08 +02:00 |
|
Enrico Ottonello
|
d6aafdf97d
|
es _id only for collection type record must be numeric format, for now we prepend 10000 to originalId value
|
2020-07-13 11:03:21 +02:00 |
|
Enrico Ottonello
|
6a804f176b
|
removed extent and audience fields, empty values cause error on portal
|
2020-07-09 16:35:04 +02:00 |
|
Enrico Ottonello
|
9358f1eaf2
|
collection data are now indexed with all required fields by es mapping
|
2020-07-08 22:00:28 +02:00 |
|
Enrico Ottonello
|
364d25f2b7
|
archeologicalResourceType.id mapped
|
2020-07-08 11:54:32 +02:00 |
|
Enrico Ottonello
|
e689623994
|
all ES fields are now mapped
|
2020-07-08 10:50:14 +02:00 |
|
Enrico Ottonello
|
9737184b1e
|
start aligning es json fields to es mapping
|
2020-07-07 13:39:22 +02:00 |
|
Enrico Ottonello
|
db9b70feb0
|
new node and workflow to index on ES
|
2020-06-16 02:36:16 +02:00 |
|
Enrico Ottonello
|
42c3f5e885
|
added indexing of collection data with dedicated query
|
2020-06-15 12:10:59 +02:00 |
|
Enrico Ottonello
|
06995feefa
|
added rest method indexOnES(datasource,collectionId) to retrieve records data from graphDB and index to ES
|
2020-06-12 18:14:41 +02:00 |
|
Enrico Ottonello
|
064f298329
|
added query to retrieve record ids collection; datasource, collection id and record id are now paramters on a query template from file
|
2020-06-11 18:20:42 +02:00 |
|
Enrico Ottonello
|
b80274fff3
|
added graphDB reader and ES indexer
|
2020-06-10 19:39:53 +02:00 |
|
Enrico Ottonello
|
9814069fb2
|
new rest method to import data into graphDB from url
|
2020-05-30 17:21:03 +02:00 |
|
Enrico Ottonello
|
30b3fa2140
|
New JobNode and workflow to enrich content on GraphDB
|
2020-05-29 16:19:55 +02:00 |
|
Enrico Ottonello
|
739dcc6b82
|
added rest method to execute sparql insert query on graphdb
|
2020-05-27 23:07:08 +02:00 |
|
Enrico Ottonello
|
226cdde77d
|
all records related to an api are published into the same graph
|
2020-02-19 14:33:54 +01:00 |
|
Enrico Ottonello
|
ff357e728b
|
modified context properties name because of update to springboot 2.1.3
|
2020-01-29 16:43:25 +01:00 |
|
Enrico Ottonello
|
019bbd2f6a
|
new classes to graphdb querying via sparql exewcution, data retrieving, feeding to elastic search
|
2020-01-28 16:53:59 +01:00 |
|
Enrico Ottonello
|
17552bafa0
|
foreach published api, the datasource name is stored into graph datasourceApis
|
2020-01-24 10:52:04 +01:00 |
|
Enrico Ottonello
|
15cc87456c
|
before any api publishing all named graph related to previous aggregation for that api are dropped
|
2020-01-15 12:30:18 +01:00 |
|
Enrico Ottonello
|
8530e488d0
|
all records related to a datasource api are now published on graphdb into n named graphs because of out of memory issue
|
2020-01-14 16:55:45 +01:00 |
|
Enrico Ottonello
|
b3d8a9193b
|
moved graphdb repository name and other properties in application.properties
|
2019-12-16 14:46:42 +01:00 |
|
Enrico Ottonello
|
3f8a4e9a46
|
new rest method to save datasourceApi information after records publishing on graphDB
|
2019-12-13 14:53:15 +01:00 |
|
Enrico Ottonello
|
2f5fb6fcb5
|
new module for graphdb publishing
|
2019-12-12 12:58:30 +01:00 |