D-Net
/
AriadnePlus
4
0
Fork 0
Code Issues Pull Requests 1 Releases Wiki Activity
588 Commits 2 Branches 1 Tag 2.6 MiB
Commit Graph

471 Commits

Author SHA1 Message Date
Enrico Ottonello 5d9ce6a5fd fix geopoint on collection 2021-11-08 16:56:28 +01:00
Enrico Ottonello 66a5677e88 added record for testing normalizer on dating filter 2021-10-27 09:42:30 +02:00
Enrico Ottonello 9d778bf75b Merge branch 'master' of https://code-repo.d4science.org/D-Net/AriadnePlus 2021-10-26 17:57:40 +02:00
Enrico Ottonello c7673f7ba1 lowercase to period name on native and temporal fields 2021-10-26 17:49:22 +02:00
Alessia Bardi af33f3ce0e provenance tripels mapped to PAV ontology 2021-10-25 15:24:59 +02:00
Alessia Bardi 0d0ebc4d77 testing DIME 2021-10-25 15:24:23 +02:00
Enrico Ottonello 0010458a23 modified log level for a few message 2021-10-22 12:32:04 +02:00
Enrico Ottonello 4a4da01d6b added titlecase on temporal and native period name 2021-10-22 11:38:19 +02:00
Enrico Ottonello 0f7e7de918 handling for title as elastic search object with text and language fields, same handling for description 2021-10-07 11:55:57 +02:00
Enrico Ottonello 0693645c68 added precision information to spatial data 2021-10-04 15:13:16 +02:00
Enrico Ottonello 25a6433809 case fix for native and derived subject 2021-09-29 17:02:45 +02:00
Enrico Ottonello 905226390c test class merged 
Merge branch 'master' of https://code-repo.d4science.org/D-Net/AriadnePlus
 2021-09-10 17:15:25 +02:00
Enrico Ottonello 05b685e0aa fixed concept label case problem and limited to 10 the about list elements retrieved 2021-09-10 17:15:21 +02:00
Alessia Bardi 5d2dec864c use read only connection 2021-09-10 15:44:34 +02:00
Enrico Ottonello f4e15dbf89 graphdb now handles request from external api client only on https 2021-09-10 12:40:59 +02:00
Enrico Ottonello acc04b0ace merged test cases 2021-09-08 17:02:16 +02:00
Enrico Ottonello c5241e3181 updated major version after merging with new_es_mapping branch 2021-09-08 16:06:30 +02:00
Enrico Ottonello 3510d7d1be Merge branch 'new_es_mapping' 
# Conflicts:
#	dnet-ariadneplus-graphdb-publisher/src/main/resources/eu/dnetlib/ariadneplus/sparql/read_collection_data_template.sparql
#	dnet-ariadneplus-graphdb-publisher/src/main/resources/eu/dnetlib/ariadneplus/sparql/read_record_data_template.sparql
#	dnet-ariadneplus-graphdb-publisher/test/java/eu/dnetlib/ariadneplus/GraphDbReaderAndESIndexTest.java
 2021-09-08 15:56:12 +02:00
Enrico Ottonello 8a68c098fb added creator enrichment for nara 2021-09-08 15:22:03 +02:00
Enrico Ottonello 0689ae6b08 enrichment for conicet records 2021-09-07 12:40:07 +02:00
Enrico Ottonello 5e4b1a87b9 fix periodo name for native period 2021-09-07 12:39:43 +02:00
Enrico Ottonello 61abf9b51e fix native period informations for collection 2021-09-06 16:39:19 +02:00
Enrico Ottonello 239d7afc41 added periodo matching rule for nara 2021-09-06 15:42:01 +02:00
Enrico Ottonello 5f4e1e2819 initial enrichment for conicet 2021-09-06 13:40:31 +02:00
Enrico Ottonello e106393e07 restored query template used for retrieving record from graphDB 2021-09-01 17:21:22 +02:00
Alessia Bardi ac01750580 ignore test that uses remote db 2021-08-26 18:00:37 +02:00
Alessia Bardi d169e1fad8 Merge branch 'master' of https://code-repo.d4science.org/D-Net/AriadnePlus 2021-08-26 17:59:35 +02:00
Alessia Bardi ea3b749792 test with read-only mode connection 2021-08-26 17:59:32 +02:00
Enrico Ottonello c2e9a4650d patched aat mappings original concept uri inside aat graph 2021-08-12 17:04:06 +02:00
Enrico Ottonello 6c6cfcf648 added inheritance from a few collection fields 2021-08-10 16:34:49 +02:00
Enrico Ottonello 087b240e05 enrichment for all infn api 2021-08-10 12:23:49 +02:00
Enrico Ottonello 4cae23f264 enrichment for nara 2021-08-09 14:41:08 +02:00
Enrico Ottonello 6789abca49 enrichment for infn torino 2021-08-09 14:40:49 +02:00
Enrico Ottonello 42eb9604a5 new class to handle publish on graphdb operation, transaction throws OOM so currently one write operation at time is executed without transaction 2021-07-27 14:58:00 +02:00
Enrico Ottonello 405f623a4c added aat mappings 2021-07-23 12:05:13 +02:00
Enrico Ottonello d584b932e8 removed temporal propagation for fieldwork on happens_during relation 2021-07-23 12:04:49 +02:00
Enrico Ottonello 9e17c18c2f updated graphdb-free-runtime to version 9.8.0, updated publisher version 2021-07-20 12:43:09 +02:00
Enrico Ottonello 21c6eb06fd fix polygon retrieving for dans and snd 2021-07-16 18:58:48 +02:00
Enrico Ottonello 9a3d380186 graphdb reader test 2021-07-07 12:37:55 +02:00
Enrico Ottonello 8944977a17 added digitalImage support 2021-06-28 12:50:39 +02:00
Enrico Ottonello 6c8c9fcba3 added fields has_type and is_about 2021-06-23 14:56:25 +02:00
Enrico Ottonello 4882e38d64 added new fields from es mapping: agent (homepage,institution,agentIdentifier) and wasCreated 2021-06-23 11:26:56 +02:00
Enrico Ottonello 89f309feaa added mapping for polygon data when aocat:has_polygonal_representation is in wkt format (snd::zip) 2021-06-22 11:06:56 +02:00
Enrico Ottonello 41896d312b fix for from and until values on temporal and native period fields 2021-06-21 15:20:39 +02:00
Enrico Ottonello 54b76c79a0 added polygon support; the correct order of the geopoints describing the polygon is needed 2021-06-18 19:32:12 +02:00
Enrico Ottonello 8ed4b8b08d added spatial boundingbox support using a 4 sides polygon wkt format 2021-06-17 17:59:51 +02:00
Alessia Bardi 9598827e1a enrichment SPARQL commands divided for FASTI collection and records, since they come from different APIs anyway and, therefore, they belong to different named graphs 2021-06-16 17:47:23 +02:00
Enrico Ottonello e91d82f32c model classes according to new es mapping; creation and indexing of a record with geopoint data 2021-06-15 23:34:59 +02:00
Alessia Bardi c60a41aa32 Aarhus DIME: enrichment queries merged into one single file 2021-06-04 17:31:28 +02:00
Alessia Bardi e0d8a0ac11 Inrap DOLIA: enrich query with AAT merged into one file 2021-06-04 15:58:55 +02:00
Alessia Bardi 699dc0c214 updated enrichment query for Inrap Dolia 2021-06-04 14:22:30 +02:00
Alessia Bardi cd8d6cdc54 INRAP draft enrich queries for Dolia 2021-06-04 14:15:52 +02:00
Alessia Bardi ae9f8e23ce now really moved to proper folder 2021-06-04 12:07:11 +02:00
Alessia Bardi 15941c029e move Aarhus enrichment queries to the right folder 2021-06-04 12:03:45 +02:00
Alessia Bardi 42ee40eff0 Merge branch 'master' of https://code-repo.d4science.org/D-Net/AriadnePlus 2021-06-03 18:51:21 +02:00
Alessia Bardi c72c7eac3f updated template for default additions - provided record 2021-06-03 18:51:10 +02:00
Alessia Bardi 1b9e7cbc67 enrich SPARQL statement for DIME (Aarhus University) 2021-06-03 18:31:55 +02:00
Enrico Ottonello 1083d4e723 set new elastic search index on staging 2021-05-31 16:44:34 +02:00
Enrico Ottonello 1545f3900f get specific error for elastic search record parsing error 2021-05-31 16:43:21 +02:00
Enrico Ottonello a9a0526bcf sparql insert for pp 2021-05-27 22:16:39 +02:00
Enrico Ottonello bd44bc6fa5 sparql insert for road 2021-05-26 18:50:11 +02:00
Enrico Ottonello 495d617c23 optimized query for retrieving collection using named graph 2021-04-22 18:11:27 +02:00
Enrico Ottonello 698828dcfc enrichment for arup collection record 2021-04-22 16:52:17 +02:00
Enrico Ottonello 49d698a532 optimized query using named graph 2021-04-22 16:51:44 +02:00
Enrico Ottonello ce8bca3a44 patch aat mapping triples replacing %2F with / (ticket #21027) 2021-04-16 17:14:31 +02:00
Enrico Ottonello 3e6a07e57c added name for contributor if missing 2021-04-14 10:43:43 +02:00
Enrico Ottonello 86d2b8d7a0 has_ARIADNE_subject inherited from collection only if missing 2021-04-12 16:07:51 +02:00
Enrico Ottonello 5682cc4bd2 added multiple descriptions for arup 2021-04-02 12:46:07 +02:00
Enrico Ottonello a3cb6201dc added periodo query to rockart 2021-04-02 12:43:59 +02:00
Enrico Ottonello 811cd03fdc sparql insert for snd rockart 2021-03-17 10:11:52 +01:00
Enrico Ottonello 4878d2131e sparql insert for rockart collection 2021-03-15 11:00:50 +01:00
Enrico Ottonello d77a1d86a4 enrichment for collectionInfo must be executed before records enrichment 2021-03-12 12:38:44 +01:00
Enrico Ottonello 742bd6fa96 added bounding box handling to configuration 2021-03-11 13:13:50 +01:00
Enrico Ottonello 2e1ec740f5 addictional sparql insert for native subjects for ads archive 2021-03-11 12:22:05 +01:00
Enrico Ottonello f2a695e9a4 ads archives sparql insert 2021-03-09 12:45:01 +01:00
Enrico Ottonello 725c596213 aligned temporal data retrieving for collection with working record one 2021-03-05 12:34:15 +01:00
Enrico Ottonello ae6abb9873 fix has_native_subject propagation 2021-03-05 11:28:57 +01:00
Enrico Ottonello f2425ffa2d removed spatial with empty data; removed duplicates on spatial list 2021-03-04 10:59:54 +01:00
Enrico Ottonello 61f0d6877d from/until retrieved also for native period 2021-03-02 17:13:33 +01:00
Enrico Ottonello 7358dd1f1f removed periodO insert from old Ariadne periodO 2021-02-22 10:52:09 +01:00
Enrico Ottonello 510161b3df further propagation for arup document and site 2021-02-17 12:56:40 +01:00
Enrico Ottonello 84993ffaba matching periodo name (if any) overwrites native period name 2021-02-16 18:19:30 +01:00
Enrico Ottonello 2d01f321d1 added propagation of temporal data for document 2021-02-16 18:16:21 +01:00
Enrico Ottonello b86195ea71 added propagation queries for has_native_subject 2021-02-16 16:01:30 +01:00
Enrico Ottonello e5b833169b fix multiple has_spatial_coverage issue 2021-02-16 10:51:14 +01:00
Enrico Ottonello 3df2befb52 fix periodo matching with upper case 2021-02-11 11:53:55 +01:00
Enrico Ottonello 512a731527 propagation of informations from deep entities 2021-02-08 10:21:31 +01:00
Enrico Ottonello 4c7f610daa fix spatial section 2021-02-05 19:53:41 +01:00
Enrico Ottonello 7c2a037dcd fix spatial section for arup collection 2021-02-05 19:44:09 +01:00
Enrico Ottonello 7ae52c1f4b added temporal data relation 2021-02-03 11:06:09 +01:00
Enrico Ottonello 2ab94d8fc2 fix has_landing_page missing for DANS 2021-02-03 11:05:14 +01:00
Enrico Ottonello 95e4d672be fix has_name for creator, enriched with rdfs:label 2021-01-31 18:12:31 +01:00
Enrico Ottonello 18801c68cc fix language code label, now retrieved parsing has_language property value 2021-01-31 16:38:47 +01:00
Enrico Ottonello cd66308827 enrichment for DANS 2021-01-31 10:51:39 +01:00
Enrico Ottonello 963ce8b048 fix periodO from Q219 for niam; enrichment for zrc sazu zbiva 2021-01-29 23:22:16 +01:00
Enrico Ottonello a1843bdc7c enrichment for ZRC Arkas and its collection, fix for Fasti 2021-01-29 15:33:48 +01:00
Enrico Ottonello 2fc7f4ec08 enrichment for FASTI 2021-01-27 23:06:33 +01:00
Enrico Ottonello a7a1a466b8 fix year date parsing 2021-01-26 16:39:47 +01:00
Enrico Ottonello fbe8af26fa sparql insert for NIAM 2021-01-26 14:10:31 +01:00
Enrico Ottonello e990541380 amcr queries for arup datasource 2021-01-21 15:38:01 +01:00
Enrico Ottonello 8e5938777a sparql queries for fasti and niam 2021-01-19 14:54:04 +01:00
Enrico Ottonello 26c5714107 fix for periodO match 2021-01-14 16:17:54 +01:00
Enrico Ottonello fa1e888dad added uppercase on periodO skos:altLabel to successful match 2021-01-10 10:49:02 +01:00
Enrico Ottonello d183b92a0c added missing prefix to 1093 sparql insert 2021-01-09 13:23:03 +01:00
Enrico Ottonello b26351d059 added missing prefix 2021-01-09 11:27:29 +01:00
Enrico Ottonello a14c66f36e added default values for has_temporal_coverage and has_spatial_coverage 2021-01-09 00:53:33 +01:00
Enrico Ottonello 9565265f3d added Rock Art mapping; fixed Site/monument mapping 2021-01-08 15:32:46 +01:00
Enrico Ottonello 5e8c38cc0b removed include because of default log to /tmp/spring.log 2020-12-22 19:33:40 +01:00
Enrico Ottonello 8a341add4c sparql insert for 1972 2020-12-01 11:29:37 +01:00
Enrico Ottonello 8baf2d7941 added BST date format handling parsing, before creating es record 2020-11-30 13:27:28 +01:00
Enrico Ottonello fbc4fc8717 added has_creator field mandatory for records 2020-11-27 12:41:11 +01:00
Enrico Ottonello 4a9a5301b3 sparql insert for 304, fix for 1093 2020-11-27 11:00:03 +01:00
Enrico Ottonello f1b1f55ae5 sparql insert for 1093 2020-11-25 13:34:18 +01:00
Enrico Ottonello 1fb0a95ada added check on record mandatory field place_name 2020-11-02 14:57:54 +01:00
Enrico Ottonello d9ca483855 added a default value for record mandatory field has_spatial_coverage 2020-11-02 14:55:18 +01:00
Enrico Ottonello eb775fc787 sparql insert for ads 4 2020-10-30 18:22:17 +01:00
Enrico Ottonello 075490f4da removed duplicated log 2020-10-30 17:29:18 +01:00
Enrico Ottonello 529151828f when IRI created from periodo url is wrong, wf has to fail 2020-10-30 17:08:46 +01:00
Enrico Ottonello 7044bcdca4 sparql insert for ads 328 2020-10-28 17:04:25 +01:00
Enrico Ottonello 991df8d4d1 sparql insert for ads 324 2020-10-28 16:41:31 +01:00
Enrico Ottonello 665a717ca0 added check on identifier, before indexing 2020-10-23 12:18:58 +02:00
Enrico Ottonello 899f3eb4d9 added has_temporal_coverage check 2020-10-22 11:26:33 +02:00
Enrico Ottonello 292f3000c4 added temporal_coverage default, if not found 2020-10-22 10:45:39 +02:00
Enrico Ottonello 240be91c5e added rest method to index single record by its identifier 2020-10-19 12:20:56 +02:00
Enrico Ottonello 9174f4df14 add graphdb connection setup 2020-10-16 19:56:14 +02:00
Enrico Ottonello f88231ba18 added new methods to retrieve resource identifiers and to start indexing by resource identifiers 2020-10-16 19:37:21 +02:00
Enrico Ottonello 0e96774895 changed http request method for indexing 2020-10-15 17:09:35 +02:00
Enrico Ottonello f5cd3cdce5 sparql insert for ads 1054 2020-10-15 13:51:02 +02:00
Enrico Ottonello ba5632f783 added has_temporal_coverage default as not provided for all ads collection, because of is a mandatory field for the query 2020-10-15 12:19:29 +02:00
Enrico Ottonello 72d37b4238 sparql insert for ads 1787, 269, fix for date on 1786 2020-10-14 19:10:24 +02:00
Enrico Ottonello 57e0470aee sparql insert for ads 1788, 388 2020-10-14 17:43:55 +02:00
Enrico Ottonello cb595f2c1c sparql insert for ads 420_monument 2020-10-14 16:30:04 +02:00
Enrico Ottonello 78818738b9 sparql insert for ads 801, 1970, 276 2020-10-14 13:11:14 +02:00
Alessia Bardi 6d8750b00d Add default for time spans that are not provided by CENIEH 2020-10-13 09:37:16 +02:00
Enrico Ottonello 263c1beb7e fix date parsing on es record creation 2020-10-13 00:31:17 +02:00
Enrico Ottonello 8f3722da13 has_native_period section set optional because of notprovided case 2020-10-12 23:00:09 +02:00
Enrico Ottonello 8777d14506 replaced test record identifier with the original placeholder %record ... 2020-10-12 22:27:54 +02:00
Alessia Bardi a452133ac3 use the new values statically inserted by the construct query 2020-10-12 18:32:56 +02:00
Alessia Bardi 5329110d93 fixed properties 2020-10-12 18:32:21 +02:00
Alessia Bardi 1b6c989f13 fixed INSERTs for CENIEH 2020-10-12 18:18:20 +02:00
Alessia Bardi 8e2b8922f6 SND native subject in upper case to enable AAT enrichment 2020-10-12 18:17:45 +02:00
Alessia Bardi 83ecd94c6f uncommented logs and set them in debug mode 2020-10-12 18:15:15 +02:00
Alessia Bardi 031f4d92f5 set explicit type to distinguish records and collections: property aocat:has_type can have multiple values 2020-10-12 18:12:41 +02:00
Miriam Baglioni d8133be6d4 insert for CENIEH 2020-10-12 15:25:16 +02:00
Enrico Ottonello ab0d99f503 added periodO uri info to retrieved collection data 2020-10-12 11:04:19 +02:00
Enrico Ottonello 0867cf3ab2 fix from e until for collection 2020-10-11 19:37:56 +02:00
Enrico Ottonello 2e732a3b5b sparql insert for ads 1;replaced AsynchJobNode with SimpleJobNode to 404 problem 2020-10-10 18:05:02 +02:00
Enrico Ottonello 2aafbc8506 fix loop condition 2020-10-10 01:03:41 +02:00
Enrico Ottonello fe614ce2ed added retry loop on HTTPQueryEvaluationException (Heap space almost full) 2020-10-10 00:55:26 +02:00
Enrico Ottonello f3c93400a3 sparql insert for ads 1785, was_issued and was_modified check has to be restricted using 1785, because of 1785 and 1786 share the same collection 2020-10-09 20:14:36 +02:00
Enrico Ottonello c83c8bf931 sparql insert for ads 1786 2020-10-09 17:47:53 +02:00
Alessia Bardi a9749ffbf2 INSERT for SND zip API - preliminary 2020-10-09 15:33:57 +02:00
Alessia Bardi a0b3e27645 updated construct query for temporals 2020-10-09 11:36:05 +02:00
Enrico Ottonello 47f46009e1 sparql insert for ads 272_event 2020-10-06 23:29:42 +02:00
Enrico Ottonello c2d13f1f4d sparql insert for ads 292 2020-10-06 22:57:18 +02:00
Enrico Ottonello 101b6261f1 sparql insert for ads 367 2020-10-06 15:51:11 +02:00
Enrico Ottonello ab3086c70c sparql insert for ads 3 2020-10-06 13:14:29 +02:00
Enrico Ottonello 8612438e86 added native subject and temporal coverage generation from about section 2020-10-05 16:45:17 +02:00
Enrico Ottonello e0729986f8 added temporal coverage generation from about section 2020-10-05 12:55:53 +02:00
Enrico Ottonello d65252873e added native_subject generation from about section 2020-10-05 11:08:00 +02:00
Enrico Ottonello 95270ef3f7 sparql insert for ads 321 2020-10-02 12:08:06 +02:00
Enrico Ottonello 05c3682e0d fix HNM periodo graph creation with all needed information, fix HNM periodoplus graph creation retrieving from/until date in year format; 858 sparql insert queries 2020-10-01 15:01:52 +02:00
Enrico Ottonello 0d44e18674 added optional fields for collection 2020-09-30 12:30:57 +02:00
Enrico Ottonello e5d0fa5254 added has_native_subject insert with uppercase; added aatplus graph creation 2020-09-29 23:13:07 +02:00
Enrico Ottonello e36e1a0080 enriched query for 420_event collection 2020-09-25 15:14:14 +02:00
Enrico Ottonello 142a296166 changed exception type, if collection indexing failes 2020-09-24 12:53:08 +02:00
Enrico Ottonello 814c908663 collection is now indexed before records, if there is an error on collection indexing, wf failed immediately; enriched query for 397 collection 2020-09-24 12:35:59 +02:00
Enrico Ottonello 12bbcdd8be Merge branch 'master' of https://code-repo.d4science.org/D-Net/AriadnePlus 2020-09-24 11:16:55 +02:00
Enrico Ottonello e5e7d85996 added ns declaration in template; enriched query for 1091 collection 2020-09-24 11:16:51 +02:00
Alessia Bardi ca3818695e ignore test that uses remote services/resources 2020-09-22 16:40:51 +02:00
Alessia Bardi f0509be5aa Merge branch 'master' of https://code-repo.d4science.org/D-Net/AriadnePlus 2020-09-22 16:28:50 +02:00
Enrico Ottonello 6976a35611 has_spatial_coverage data set to optional for collection 2020-09-18 17:24:36 +02:00
Enrico Ottonello b4476f6ee7 sparql insert set for 272_monument; test for retrieving collection records for 272_monument, that does not retrieve any records, because of spatial_region data are not found 2020-09-17 16:32:43 +02:00
Alessia Bardi ea4ed22b4d Collection 272 does not exist, we have 272_monument and 272_event 2020-09-15 15:21:07 +02:00
Alessia Bardi be31a772a1 SPARQL INSERT for HNM 2020-09-07 15:46:07 +02:00
Alessia Bardi 12c80713d7 indexing result per record logged in debug mode instead of info 2020-09-07 15:45:26 +02:00
Alessia Bardi 832b95f064 parse must succeed if the date is already in the expected format 2020-09-07 15:37:40 +02:00
Alessia Bardi 1e3ff13aa4 test date parsing 2020-09-07 15:37:05 +02:00
Alessia Bardi 630cf59022 test date parsing 2020-09-07 15:36:30 +02:00
Alessia Bardi 38a2ab747e test 2020-09-07 15:01:00 +02:00
Alessia Bardi 19ebb63d88 fixed log 2020-09-07 15:00:09 +02:00
Alessia Bardi f1e3de1b07 avoid null pointer exception 2020-09-07 14:59:54 +02:00
Alessia Bardi 17f87efd1f test 2020-09-07 14:58:39 +02:00
Alessia Bardi 4fb5bb2941 it is legal to have no individual resources or no collections: let's index what we have 2020-09-04 16:57:38 +02:00
Alessia Bardi f55699a324 sparql insert for HNM (AAT missing) 2020-09-04 16:56:52 +02:00
Alessia Bardi 420e31d866 preparing sparql inserts for ADS 271 and 272 2020-09-04 16:56:17 +02:00
Alessia Bardi e63f25217c Insert defaults for collections of HNM 2020-09-04 11:29:01 +02:00
Alessia Bardi ca2f6dbb21 updated labels 2020-09-04 11:26:06 +02:00
Alessia Bardi 1b11d0ee1c default for collection has_type (needed by HNM) 2020-09-04 11:11:22 +02:00
Alessia Bardi 7338a759c8 removed duplicate check on access rights 2020-09-04 10:57:11 +02:00
Enrico Ottonello db2ad3f97e multithreads http client not needed on indexjobnode, maybe avoid 404 response after a few minutes, added log on each indexing operation 2020-08-12 15:47:55 +02:00
Alessia Bardi 56088d7779 need to add info from the collSubjects graph -- ARIADNE subject and collection id 2020-08-07 18:53:15 +02:00
Alessia Bardi ca42e0d01e fixed SPARQL INSERT for PeriodO enrichment 2020-08-07 17:45:45 +02:00
Alessia Bardi 23f1feb44a updated SPARQL inserts for ADS 2020-08-07 16:41:07 +02:00
Alessia Bardi 8329f6e9dc useless if removed 2020-08-07 16:40:21 +02:00
Alessia Bardi c49d6ce109 added query for spatial region of collections 2020-08-07 16:24:15 +02:00
Alessia Bardi 02c5014fd5 removed redundand isPart 2020-08-07 16:23:26 +02:00
Alessia Bardi e8d65287cc check additional mandatory fields 2020-08-07 16:22:19 +02:00
Alessia Bardi 3e2e9ff372 Insert default creator also for collections 2020-08-07 16:21:47 +02:00
Alessia Bardi d0372f6f5f ignore test that uses remote resources 2020-08-07 14:57:01 +02:00
Alessia Bardi ca4ad7ea3b Need to split the multiple inserts and perform separate transactions in GraphDB Free edition. See https://stackoverflow.com/questions/54428161/graphdb-read-check-and-update-in-a-transaction 2020-08-07 14:54:15 +02:00
Alessia Bardi 587887abd6 sparql inserts for ADS 836 2020-08-07 14:53:21 +02:00
Alessia Bardi 2c282d31b2 sparql inserts for ADS 836 2020-08-07 14:51:56 +02:00
Alessia Bardi 223c304702 sparql inserts for ADS 273 2020-08-07 12:57:21 +02:00
Alessia Bardi 69ca7eaade removed redundand lines of code 2020-08-07 12:44:36 +02:00
Alessia Bardi b3f5f38c02 sparql query to get the records without mandatory fields 2020-08-07 12:31:41 +02:00
Alessia Bardi f8a3bc5a0e sparql query to get the collections without mandatory fields 2020-08-07 12:28:17 +02:00
Alessia Bardi e82043d9eb added comments 2020-08-07 11:45:09 +02:00
Alessia Bardi 7040ad741b moved SPARQL query for ADS 270 in dedicated folder 2020-08-06 20:29:06 +02:00
Alessia Bardi 6365fa583d sparql templates for the different fix and enrichment phases 2020-08-06 20:28:16 +02:00
Alessia Bardi cf2f2e6f8b organised templates and prepared folders for ADS SPARQL inserts 2020-08-06 20:26:01 +02:00
Enrico Ottonello 54458be7f5 order in sparql insert execution is important, this query contains all sparql insert in the correct order 2020-07-29 16:37:22 +02:00
Enrico Ottonello 220c6be6db report returned to wf node 2020-07-24 13:15:27 +02:00
Enrico Ottonello 214502cc11 removed test publisher 2020-07-24 12:59:59 +02:00
Enrico Ottonello f61f794c25 removed a few logs 2020-07-24 12:34:06 +02:00
Enrico Ottonello 11e00c806d reset json parser status on each record 2020-07-24 01:48:17 +02:00
Enrico Ottonello b29b4e1fd7 added log on json parser 2020-07-24 01:21:25 +02:00
Enrico Ottonello eb61aebcd5 added other check on parsing json from rdf statements 2020-07-24 00:32:49 +02:00
Enrico Ottonello 5eaa655f01 added other check on es return code 2020-07-23 23:12:07 +02:00
Enrico Ottonello 173b0c289e to avoid overload on graphdb heap it is better to slowing down query execution 2020-07-22 23:47:08 +02:00
Enrico Ottonello 5bb65a8854 added response code from es; added sparql insert for copying ariadne_subject value into collection for ads; fix sparql insert for copying collection data into records, collection id condition is needed 2020-07-22 23:13:47 +02:00
Enrico Ottonello 766f8199f2 removed empty values/structures from record/collection to es 2020-07-17 17:42:08 +02:00
Enrico Ottonello d6aafdf97d es _id only for collection type record must be numeric format, for now we prepend 10000 to originalId value 2020-07-13 11:03:21 +02:00
Enrico Ottonello 6a804f176b removed extent and audience fields, empty values cause error on portal 2020-07-09 16:35:04 +02:00
Enrico Ottonello 31b7c61f62 Merge branch 'master' of https://code-repo.d4science.org/D-Net/AriadnePlus 2020-07-08 22:00:33 +02:00
Enrico Ottonello 9358f1eaf2 collection data are now indexed with all required fields by es mapping 2020-07-08 22:00:28 +02:00
Alessia Bardi eb805ebf9b test for checking structural equality in new and old records in elasticsearch 2020-07-08 16:34:48 +02:00
Alessia Bardi 7b644e04da added dep for json diff 2020-07-08 16:34:03 +02:00
Alessia Bardi 4cf842f906 test for checking structural equality in new and old records in elasticsearch 2020-07-08 16:33:40 +02:00
Enrico Ottonello 364d25f2b7 archeologicalResourceType.id mapped 2020-07-08 11:54:32 +02:00
Enrico Ottonello e689623994 all ES fields are now mapped 2020-07-08 10:50:14 +02:00
Enrico Ottonello 9737184b1e start aligning es json fields to es mapping 2020-07-07 13:39:22 +02:00
Enrico Ottonello db9b70feb0 new node and workflow to index on ES 2020-06-16 02:36:16 +02:00
Enrico Ottonello 42c3f5e885 added indexing of collection data with dedicated query 2020-06-15 12:10:59 +02:00
Enrico Ottonello 06995feefa added rest method indexOnES(datasource,collectionId) to retrieve records data from graphDB and index to ES 2020-06-12 18:14:41 +02:00
Enrico Ottonello 064f298329 added query to retrieve record ids collection; datasource, collection id and record id are now paramters on a query template from file 2020-06-11 18:20:42 +02:00
Enrico Ottonello b80274fff3 added graphDB reader and ES indexer 2020-06-10 19:39:53 +02:00
Enrico Ottonello fa1d268b36 added query to link data from collection level to records level 2020-06-10 11:53:37 +02:00
Enrico Ottonello af2eb7bb91 added sparql query folder 2020-06-09 11:44:48 +02:00
Enrico Ottonello aa1c6c2fda added query to generate aat and periodo enriched graphs 2020-06-09 11:39:56 +02:00
Enrico Ottonello 9814069fb2 new rest method to import data into graphDB from url 2020-05-30 17:21:03 +02:00
Enrico Ottonello 30b3fa2140 New JobNode and workflow to enrich content on GraphDB 2020-05-29 16:19:55 +02:00
Enrico Ottonello 739dcc6b82 added rest method to execute sparql insert query on graphdb 2020-05-27 23:07:08 +02:00
Enrico Ottonello d23f08f750 removed BIND not necessary 2020-04-03 15:53:52 +02:00
Enrico Ottonello a005328dfe fix resource type variable name 2020-03-31 11:10:41 +02:00
Enrico Ottonello d52ad32f68 added narrow and broad for aat match 2020-03-20 17:46:55 +01:00
Enrico Ottonello 7ae6d400b6 temporal coverage joined with periodO data that must be directly imported in graphDb in turtle format together with periodO mapping 2020-03-20 13:01:54 +01:00
Enrico Ottonello ef42eb551a fix spatial subselect: region, region point and region bounding box data must be referenced by 3 different variables inside the query 2020-03-17 16:27:14 +01:00
Enrico Ottonello 32b91c277d all not madatory sets of conditions must be set as optional 2020-03-16 19:16:42 +01:00
Enrico Ottonello 5c3d50f309 originalId set to mandatory 2020-03-06 13:11:35 +01:00