Commit Graph

85 Commits

Author SHA1 Message Date
Enrico Ottonello 2e1ec740f5 addictional sparql insert for native subjects for ads archive 2021-03-11 12:22:05 +01:00
Enrico Ottonello 725c596213 aligned temporal data retrieving for collection with working record one 2021-03-05 12:34:15 +01:00
Enrico Ottonello f2425ffa2d removed spatial with empty data; removed duplicates on spatial list 2021-03-04 10:59:54 +01:00
Enrico Ottonello 61f0d6877d from/until retrieved also for native period 2021-03-02 17:13:33 +01:00
Enrico Ottonello 510161b3df further propagation for arup document and site 2021-02-17 12:56:40 +01:00
Enrico Ottonello 84993ffaba matching periodo name (if any) overwrites native period name 2021-02-16 18:19:30 +01:00
Enrico Ottonello a7a1a466b8 fix year date parsing 2021-01-26 16:39:47 +01:00
Enrico Ottonello 8baf2d7941 added BST date format handling parsing, before creating es record 2020-11-30 13:27:28 +01:00
Enrico Ottonello 263c1beb7e fix date parsing on es record creation 2020-10-13 00:31:17 +02:00
Enrico Ottonello 8f3722da13 has_native_period section set optional because of notprovided case 2020-10-12 23:00:09 +02:00
Alessia Bardi 031f4d92f5 set explicit type to distinguish records and collections: property aocat:has_type can have multiple values 2020-10-12 18:12:41 +02:00
Enrico Ottonello ab0d99f503 added periodO uri info to retrieved collection data 2020-10-12 11:04:19 +02:00
Alessia Bardi ca3818695e ignore test that uses remote services/resources 2020-09-22 16:40:51 +02:00
Enrico Ottonello b4476f6ee7 sparql insert set for 272_monument; test for retrieving collection records for 272_monument, that does not retrieve any records, because of spatial_region data are not found 2020-09-17 16:32:43 +02:00
Alessia Bardi 1e3ff13aa4 test date parsing 2020-09-07 15:37:05 +02:00
Alessia Bardi 630cf59022 test date parsing 2020-09-07 15:36:30 +02:00
Alessia Bardi 38a2ab747e test 2020-09-07 15:01:00 +02:00
Alessia Bardi 17f87efd1f test 2020-09-07 14:58:39 +02:00
Enrico Ottonello db2ad3f97e multithreads http client not needed on indexjobnode, maybe avoid 404 response after a few minutes, added log on each indexing operation 2020-08-12 15:47:55 +02:00
Alessia Bardi d0372f6f5f ignore test that uses remote resources 2020-08-07 14:57:01 +02:00
Enrico Ottonello eb61aebcd5 added other check on parsing json from rdf statements 2020-07-24 00:32:49 +02:00
Enrico Ottonello 5bb65a8854 added response code from es; added sparql insert for copying ariadne_subject value into collection for ads; fix sparql insert for copying collection data into records, collection id condition is needed 2020-07-22 23:13:47 +02:00
Enrico Ottonello 766f8199f2 removed empty values/structures from record/collection to es 2020-07-17 17:42:08 +02:00
Enrico Ottonello d6aafdf97d es _id only for collection type record must be numeric format, for now we prepend 10000 to originalId value 2020-07-13 11:03:21 +02:00
Enrico Ottonello 6a804f176b removed extent and audience fields, empty values cause error on portal 2020-07-09 16:35:04 +02:00
Enrico Ottonello 31b7c61f62 Merge branch 'master' of https://code-repo.d4science.org/D-Net/AriadnePlus 2020-07-08 22:00:33 +02:00
Enrico Ottonello 9358f1eaf2 collection data are now indexed with all required fields by es mapping 2020-07-08 22:00:28 +02:00
Alessia Bardi eb805ebf9b test for checking structural equality in new and old records in elasticsearch 2020-07-08 16:34:48 +02:00
Alessia Bardi 4cf842f906 test for checking structural equality in new and old records in elasticsearch 2020-07-08 16:33:40 +02:00
Enrico Ottonello 9737184b1e start aligning es json fields to es mapping 2020-07-07 13:39:22 +02:00
Enrico Ottonello db9b70feb0 new node and workflow to index on ES 2020-06-16 02:36:16 +02:00
Enrico Ottonello 06995feefa added rest method indexOnES(datasource,collectionId) to retrieve records data from graphDB and index to ES 2020-06-12 18:14:41 +02:00
Enrico Ottonello 064f298329 added query to retrieve record ids collection; datasource, collection id and record id are now paramters on a query template from file 2020-06-11 18:20:42 +02:00
Enrico Ottonello b80274fff3 added graphDB reader and ES indexer 2020-06-10 19:39:53 +02:00
Enrico Ottonello 019bbd2f6a new classes to graphdb querying via sparql exewcution, data retrieving, feeding to elastic search 2020-01-28 16:53:59 +01:00