Commit Graph

71 Commits

Author SHA1 Message Date
Enrico Ottonello 6c8c9fcba3 added fields has_type and is_about 2021-06-23 14:56:25 +02:00
Enrico Ottonello 4882e38d64 added new fields from es mapping: agent (homepage,institution,agentIdentifier) and wasCreated 2021-06-23 11:26:56 +02:00
Enrico Ottonello 89f309feaa added mapping for polygon data when aocat:has_polygonal_representation is in wkt format (snd::zip) 2021-06-22 11:06:56 +02:00
Enrico Ottonello 41896d312b fix for from and until values on temporal and native period fields 2021-06-21 15:20:39 +02:00
Enrico Ottonello 54b76c79a0 added polygon support; the correct order of the geopoints describing the polygon is needed 2021-06-18 19:32:12 +02:00
Enrico Ottonello 8ed4b8b08d added spatial boundingbox support using a 4 sides polygon wkt format 2021-06-17 17:59:51 +02:00
Enrico Ottonello e91d82f32c model classes according to new es mapping; creation and indexing of a record with geopoint data 2021-06-15 23:34:59 +02:00
Enrico Ottonello 1545f3900f get specific error for elastic search record parsing error 2021-05-31 16:43:21 +02:00
Enrico Ottonello 5682cc4bd2 added multiple descriptions for arup 2021-04-02 12:46:07 +02:00
Enrico Ottonello f2425ffa2d removed spatial with empty data; removed duplicates on spatial list 2021-03-04 10:59:54 +01:00
Enrico Ottonello 84993ffaba matching periodo name (if any) overwrites native period name 2021-02-16 18:19:30 +01:00
Enrico Ottonello e5b833169b fix multiple has_spatial_coverage issue 2021-02-16 10:51:14 +01:00
Enrico Ottonello a7a1a466b8 fix year date parsing 2021-01-26 16:39:47 +01:00
Enrico Ottonello 9565265f3d added Rock Art mapping; fixed Site/monument mapping 2021-01-08 15:32:46 +01:00
Enrico Ottonello 8baf2d7941 added BST date format handling parsing, before creating es record 2020-11-30 13:27:28 +01:00
Enrico Ottonello 075490f4da removed duplicated log 2020-10-30 17:29:18 +01:00
Enrico Ottonello 529151828f when IRI created from periodo url is wrong, wf has to fail 2020-10-30 17:08:46 +01:00
Enrico Ottonello 665a717ca0 added check on identifier, before indexing 2020-10-23 12:18:58 +02:00
Enrico Ottonello 240be91c5e added rest method to index single record by its identifier 2020-10-19 12:20:56 +02:00
Enrico Ottonello 9174f4df14 add graphdb connection setup 2020-10-16 19:56:14 +02:00
Enrico Ottonello f88231ba18 added new methods to retrieve resource identifiers and to start indexing by resource identifiers 2020-10-16 19:37:21 +02:00
Enrico Ottonello 0e96774895 changed http request method for indexing 2020-10-15 17:09:35 +02:00
Enrico Ottonello 263c1beb7e fix date parsing on es record creation 2020-10-13 00:31:17 +02:00
Alessia Bardi a452133ac3 use the new values statically inserted by the construct query 2020-10-12 18:32:56 +02:00
Alessia Bardi 83ecd94c6f uncommented logs and set them in debug mode 2020-10-12 18:15:15 +02:00
Enrico Ottonello 2aafbc8506 fix loop condition 2020-10-10 01:03:41 +02:00
Enrico Ottonello fe614ce2ed added retry loop on HTTPQueryEvaluationException (Heap space almost full) 2020-10-10 00:55:26 +02:00
Enrico Ottonello 142a296166 changed exception type, if collection indexing failes 2020-09-24 12:53:08 +02:00
Enrico Ottonello 814c908663 collection is now indexed before records, if there is an error on collection indexing, wf failed immediately; enriched query for 397 collection 2020-09-24 12:35:59 +02:00
Alessia Bardi 12c80713d7 indexing result per record logged in debug mode instead of info 2020-09-07 15:45:26 +02:00
Alessia Bardi 832b95f064 parse must succeed if the date is already in the expected format 2020-09-07 15:37:40 +02:00
Alessia Bardi 19ebb63d88 fixed log 2020-09-07 15:00:09 +02:00
Alessia Bardi f1e3de1b07 avoid null pointer exception 2020-09-07 14:59:54 +02:00
Alessia Bardi 4fb5bb2941 it is legal to have no individual resources or no collections: let's index what we have 2020-09-04 16:57:38 +02:00
Enrico Ottonello db2ad3f97e multithreads http client not needed on indexjobnode, maybe avoid 404 response after a few minutes, added log on each indexing operation 2020-08-12 15:47:55 +02:00
Alessia Bardi 8329f6e9dc useless if removed 2020-08-07 16:40:21 +02:00
Alessia Bardi ca4ad7ea3b Need to split the multiple inserts and perform separate transactions in GraphDB Free edition. See https://stackoverflow.com/questions/54428161/graphdb-read-check-and-update-in-a-transaction 2020-08-07 14:54:15 +02:00
Alessia Bardi 69ca7eaade removed redundand lines of code 2020-08-07 12:44:36 +02:00
Enrico Ottonello 220c6be6db report returned to wf node 2020-07-24 13:15:27 +02:00
Enrico Ottonello 214502cc11 removed test publisher 2020-07-24 12:59:59 +02:00
Enrico Ottonello f61f794c25 removed a few logs 2020-07-24 12:34:06 +02:00
Enrico Ottonello 11e00c806d reset json parser status on each record 2020-07-24 01:48:17 +02:00
Enrico Ottonello b29b4e1fd7 added log on json parser 2020-07-24 01:21:25 +02:00
Enrico Ottonello eb61aebcd5 added other check on parsing json from rdf statements 2020-07-24 00:32:49 +02:00
Enrico Ottonello 5eaa655f01 added other check on es return code 2020-07-23 23:12:07 +02:00
Enrico Ottonello 173b0c289e to avoid overload on graphdb heap it is better to slowing down query execution 2020-07-22 23:47:08 +02:00
Enrico Ottonello 5bb65a8854 added response code from es; added sparql insert for copying ariadne_subject value into collection for ads; fix sparql insert for copying collection data into records, collection id condition is needed 2020-07-22 23:13:47 +02:00
Enrico Ottonello 766f8199f2 removed empty values/structures from record/collection to es 2020-07-17 17:42:08 +02:00
Enrico Ottonello d6aafdf97d es _id only for collection type record must be numeric format, for now we prepend 10000 to originalId value 2020-07-13 11:03:21 +02:00
Enrico Ottonello 6a804f176b removed extent and audience fields, empty values cause error on portal 2020-07-09 16:35:04 +02:00