D-Net
/
AriadnePlus
4
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
532 Commits 4 Branches 1 Tag 2.6 MiB
Commit Graph

532 Commits

Author SHA1 Message Date
Alessia Bardi 7040ad741b moved SPARQL query for ADS 270 in dedicated folder 2020-08-06 20:29:06 +02:00
Alessia Bardi 6365fa583d sparql templates for the different fix and enrichment phases 2020-08-06 20:28:16 +02:00
Alessia Bardi cf2f2e6f8b organised templates and prepared folders for ADS SPARQL inserts 2020-08-06 20:26:01 +02:00
Enrico Ottonello 54458be7f5 order in sparql insert execution is important, this query contains all sparql insert in the correct order 2020-07-29 16:37:22 +02:00
Enrico Ottonello 5c3ef5f25b retrieved body from server response 2020-07-24 18:32:28 +02:00
Enrico Ottonello e189283059 retrieved report on index results 2020-07-24 17:59:23 +02:00
Enrico Ottonello 220c6be6db report returned to wf node 2020-07-24 13:15:27 +02:00
Enrico Ottonello 214502cc11 removed test publisher 2020-07-24 12:59:59 +02:00
Enrico Ottonello f61f794c25 removed a few logs 2020-07-24 12:34:06 +02:00
Enrico Ottonello 11e00c806d reset json parser status on each record 2020-07-24 01:48:17 +02:00
Enrico Ottonello b29b4e1fd7 added log on json parser 2020-07-24 01:21:25 +02:00
Enrico Ottonello eb61aebcd5 added other check on parsing json from rdf statements 2020-07-24 00:32:49 +02:00
Enrico Ottonello 5eaa655f01 added other check on es return code 2020-07-23 23:12:07 +02:00
Enrico Ottonello 173b0c289e to avoid overload on graphdb heap it is better to slowing down query execution 2020-07-22 23:47:08 +02:00
Enrico Ottonello 5bb65a8854 added response code from es; added sparql insert for copying ariadne_subject value into collection for ads; fix sparql insert for copying collection data into records, collection id condition is needed 2020-07-22 23:13:47 +02:00
Enrico Ottonello 766f8199f2 removed empty values/structures from record/collection to es 2020-07-17 17:42:08 +02:00
Enrico Ottonello d6aafdf97d es _id only for collection type record must be numeric format, for now we prepend 10000 to originalId value 2020-07-13 11:03:21 +02:00
Enrico Ottonello 6a804f176b removed extent and audience fields, empty values cause error on portal 2020-07-09 16:35:04 +02:00
Enrico Ottonello 31b7c61f62 Merge branch 'master' of https://code-repo.d4science.org/D-Net/AriadnePlus 2020-07-08 22:00:33 +02:00
Enrico Ottonello 9358f1eaf2 collection data are now indexed with all required fields by es mapping 2020-07-08 22:00:28 +02:00
Alessia Bardi eb805ebf9b test for checking structural equality in new and old records in elasticsearch 2020-07-08 16:34:48 +02:00
Alessia Bardi 7b644e04da added dep for json diff 2020-07-08 16:34:03 +02:00
Alessia Bardi 4cf842f906 test for checking structural equality in new and old records in elasticsearch 2020-07-08 16:33:40 +02:00
Enrico Ottonello 364d25f2b7 archeologicalResourceType.id mapped 2020-07-08 11:54:32 +02:00
Enrico Ottonello e689623994 all ES fields are now mapped 2020-07-08 10:50:14 +02:00
Enrico Ottonello 9737184b1e start aligning es json fields to es mapping 2020-07-07 13:39:22 +02:00
Alessia Bardi 74d0f440e3 Merge branch 'master' of https://code-repo.d4science.org/D-Net/AriadnePlus 2020-06-23 10:32:11 +02:00
Alessia Bardi e5d1f845a3 updated policy 2020-06-23 10:32:04 +02:00
Enrico Ottonello db9b70feb0 new node and workflow to index on ES 2020-06-16 02:36:16 +02:00
Enrico Ottonello 42c3f5e885 added indexing of collection data with dedicated query 2020-06-15 12:10:59 +02:00
Enrico Ottonello 06995feefa added rest method indexOnES(datasource,collectionId) to retrieve records data from graphDB and index to ES 2020-06-12 18:14:41 +02:00
Enrico Ottonello 064f298329 added query to retrieve record ids collection; datasource, collection id and record id are now paramters on a query template from file 2020-06-11 18:20:42 +02:00
Enrico Ottonello b80274fff3 added graphDB reader and ES indexer 2020-06-10 19:39:53 +02:00
Enrico Ottonello fa1d268b36 added query to link data from collection level to records level 2020-06-10 11:53:37 +02:00
Enrico Ottonello af2eb7bb91 added sparql query folder 2020-06-09 11:44:48 +02:00
Enrico Ottonello aa1c6c2fda added query to generate aat and periodo enriched graphs 2020-06-09 11:39:56 +02:00
Alessia Bardi 4d68ecad58 the job fails if the endpoint returns client errors 4xx 2020-06-05 16:19:54 +02:00
Alessia Bardi 30410e8cca need to enclose policy in CDATA, otherwise we get an error from the X3ML engine 'only whitespace content allowed before start tag and not C' 2020-06-05 16:02:31 +02:00
Alessia Bardi fa3707a56f updated policy 2020-06-05 15:47:25 +02:00
Alessia Bardi e5cfbd01af ADS plugin now able to process one single remote file 2020-06-05 15:20:21 +02:00
Alessia Bardi b220b9de4e fixed tests so they do not fail 2020-06-05 15:10:17 +02:00
Enrico Ottonello 819ef88520 new workflow to import periodo from url 2020-06-01 12:29:48 +02:00
Enrico Ottonello 9814069fb2 new rest method to import data into graphDB from url 2020-05-30 17:21:03 +02:00
Enrico Ottonello 30b3fa2140 New JobNode and workflow to enrich content on GraphDB 2020-05-29 16:19:55 +02:00
Enrico Ottonello 739dcc6b82 added rest method to execute sparql insert query on graphdb 2020-05-27 23:07:08 +02:00
Enrico Ottonello d23f08f750 removed BIND not necessary 2020-04-03 15:53:52 +02:00
Enrico Ottonello 996150a315 modified access protocol parameter name (mandatory for aat and periodo), it is the name of the root node, inside the metadata record, that is generated converting original json record 2020-04-03 11:58:04 +02:00
Enrico Ottonello a005328dfe fix resource type variable name 2020-03-31 11:10:41 +02:00
Enrico Ottonello d52ad32f68 added narrow and broad for aat match 2020-03-20 17:46:55 +01:00
Enrico Ottonello 7ae6d400b6 temporal coverage joined with periodO data that must be directly imported in graphDb in turtle format together with periodO mapping 2020-03-20 13:01:54 +01:00