 Enrico Ottonello
|
f3c93400a3
|
sparql insert for ads 1785, was_issued and was_modified check has to be restricted using 1785, because of 1785 and 1786 share the same collection
|
2020-10-09 20:14:36 +02:00 |
|
Enrico Ottonello
|
c83c8bf931
|
sparql insert for ads 1786
|
2020-10-09 17:47:53 +02:00 |
 Alessia Bardi
|
a9749ffbf2
|
INSERT for SND zip API - preliminary
|
2020-10-09 15:33:57 +02:00 |
|
Alessia Bardi
|
a0b3e27645
|
updated construct query for temporals
|
2020-10-09 11:36:05 +02:00 |
|
Enrico Ottonello
|
47f46009e1
|
sparql insert for ads 272_event
|
2020-10-06 23:29:42 +02:00 |
|
Enrico Ottonello
|
c2d13f1f4d
|
sparql insert for ads 292
|
2020-10-06 22:57:18 +02:00 |
|
Enrico Ottonello
|
101b6261f1
|
sparql insert for ads 367
|
2020-10-06 15:51:11 +02:00 |
|
Enrico Ottonello
|
ab3086c70c
|
sparql insert for ads 3
|
2020-10-06 13:14:29 +02:00 |
|
Enrico Ottonello
|
8612438e86
|
added native subject and temporal coverage generation from about section
|
2020-10-05 16:45:17 +02:00 |
|
Enrico Ottonello
|
e0729986f8
|
added temporal coverage generation from about section
|
2020-10-05 12:55:53 +02:00 |
|
Enrico Ottonello
|
d65252873e
|
added native_subject generation from about section
|
2020-10-05 11:08:00 +02:00 |
|
Enrico Ottonello
|
95270ef3f7
|
sparql insert for ads 321
|
2020-10-02 12:08:06 +02:00 |
|
Enrico Ottonello
|
05c3682e0d
|
fix HNM periodo graph creation with all needed information, fix HNM periodoplus graph creation retrieving from/until date in year format; 858 sparql insert queries
|
2020-10-01 15:01:52 +02:00 |
|
Enrico Ottonello
|
0d44e18674
|
added optional fields for collection
|
2020-09-30 12:30:57 +02:00 |
|
Enrico Ottonello
|
e5d0fa5254
|
added has_native_subject insert with uppercase; added aatplus graph creation
|
2020-09-29 23:13:07 +02:00 |
|
Enrico Ottonello
|
e36e1a0080
|
enriched query for 420_event collection
|
2020-09-25 15:14:14 +02:00 |
|
Enrico Ottonello
|
142a296166
|
changed exception type, if collection indexing failes
|
2020-09-24 12:53:08 +02:00 |
|
Enrico Ottonello
|
814c908663
|
collection is now indexed before records, if there is an error on collection indexing, wf failed immediately; enriched query for 397 collection
|
2020-09-24 12:35:59 +02:00 |
|
Enrico Ottonello
|
12bbcdd8be
|
Merge branch 'master' of https://code-repo.d4science.org/D-Net/AriadnePlus
|
2020-09-24 11:16:55 +02:00 |
|
Enrico Ottonello
|
e5e7d85996
|
added ns declaration in template; enriched query for 1091 collection
|
2020-09-24 11:16:51 +02:00 |
|
Alessia Bardi
|
ca3818695e
|
ignore test that uses remote services/resources
|
2020-09-22 16:40:51 +02:00 |
|
Alessia Bardi
|
f0509be5aa
|
Merge branch 'master' of https://code-repo.d4science.org/D-Net/AriadnePlus
|
2020-09-22 16:28:50 +02:00 |
|
Enrico Ottonello
|
6976a35611
|
has_spatial_coverage data set to optional for collection
|
2020-09-18 17:24:36 +02:00 |
|
Enrico Ottonello
|
b4476f6ee7
|
sparql insert set for 272_monument; test for retrieving collection records for 272_monument, that does not retrieve any records, because of spatial_region data are not found
|
2020-09-17 16:32:43 +02:00 |
|
Alessia Bardi
|
ea4ed22b4d
|
Collection 272 does not exist, we have 272_monument and 272_event
|
2020-09-15 15:21:07 +02:00 |
|
Alessia Bardi
|
be31a772a1
|
SPARQL INSERT for HNM
|
2020-09-07 15:46:07 +02:00 |
|
Alessia Bardi
|
12c80713d7
|
indexing result per record logged in debug mode instead of info
|
2020-09-07 15:45:26 +02:00 |
|
Alessia Bardi
|
832b95f064
|
parse must succeed if the date is already in the expected format
|
2020-09-07 15:37:40 +02:00 |
|
Alessia Bardi
|
1e3ff13aa4
|
test date parsing
|
2020-09-07 15:37:05 +02:00 |
|
Alessia Bardi
|
630cf59022
|
test date parsing
|
2020-09-07 15:36:30 +02:00 |
|
Alessia Bardi
|
38a2ab747e
|
test
|
2020-09-07 15:01:00 +02:00 |
|
Alessia Bardi
|
19ebb63d88
|
fixed log
|
2020-09-07 15:00:09 +02:00 |
|
Alessia Bardi
|
f1e3de1b07
|
avoid null pointer exception
|
2020-09-07 14:59:54 +02:00 |
|
Alessia Bardi
|
17f87efd1f
|
test
|
2020-09-07 14:58:39 +02:00 |
|
Alessia Bardi
|
4fb5bb2941
|
it is legal to have no individual resources or no collections: let's index what we have
|
2020-09-04 16:57:38 +02:00 |
|
Alessia Bardi
|
f55699a324
|
sparql insert for HNM (AAT missing)
|
2020-09-04 16:56:52 +02:00 |
|
Alessia Bardi
|
420e31d866
|
preparing sparql inserts for ADS 271 and 272
|
2020-09-04 16:56:17 +02:00 |
|
Alessia Bardi
|
e63f25217c
|
Insert defaults for collections of HNM
|
2020-09-04 11:29:01 +02:00 |
|
Alessia Bardi
|
ca2f6dbb21
|
updated labels
|
2020-09-04 11:26:06 +02:00 |
|
Alessia Bardi
|
1b11d0ee1c
|
default for collection has_type (needed by HNM)
|
2020-09-04 11:11:22 +02:00 |
|
Alessia Bardi
|
7338a759c8
|
removed duplicate check on access rights
|
2020-09-04 10:57:11 +02:00 |
|
Enrico Ottonello
|
db2ad3f97e
|
multithreads http client not needed on indexjobnode, maybe avoid 404 response after a few minutes, added log on each indexing operation
|
2020-08-12 15:47:55 +02:00 |
|
Alessia Bardi
|
56088d7779
|
need to add info from the collSubjects graph -- ARIADNE subject and collection id
|
2020-08-07 18:53:15 +02:00 |
|
Alessia Bardi
|
ca42e0d01e
|
fixed SPARQL INSERT for PeriodO enrichment
|
2020-08-07 17:45:45 +02:00 |
|
Alessia Bardi
|
23f1feb44a
|
updated SPARQL inserts for ADS
|
2020-08-07 16:41:07 +02:00 |
|
Alessia Bardi
|
8329f6e9dc
|
useless if removed
|
2020-08-07 16:40:21 +02:00 |
|
Alessia Bardi
|
c49d6ce109
|
added query for spatial region of collections
|
2020-08-07 16:24:15 +02:00 |
|
Alessia Bardi
|
02c5014fd5
|
removed redundand isPart
|
2020-08-07 16:23:26 +02:00 |
|
Alessia Bardi
|
e8d65287cc
|
check additional mandatory fields
|
2020-08-07 16:22:19 +02:00 |
|
Alessia Bardi
|
3e2e9ff372
|
Insert default creator also for collections
|
2020-08-07 16:21:47 +02:00 |
|
Alessia Bardi
|
d0372f6f5f
|
ignore test that uses remote resources
|
2020-08-07 14:57:01 +02:00 |
|
Alessia Bardi
|
ca4ad7ea3b
|
Need to split the multiple inserts and perform separate transactions in GraphDB Free edition. See https://stackoverflow.com/questions/54428161/graphdb-read-check-and-update-in-a-transaction
|
2020-08-07 14:54:15 +02:00 |
|
Alessia Bardi
|
587887abd6
|
sparql inserts for ADS 836
|
2020-08-07 14:53:21 +02:00 |
|
Alessia Bardi
|
2c282d31b2
|
sparql inserts for ADS 836
|
2020-08-07 14:51:56 +02:00 |
|
Alessia Bardi
|
223c304702
|
sparql inserts for ADS 273
|
2020-08-07 12:57:21 +02:00 |
|
Alessia Bardi
|
69ca7eaade
|
removed redundand lines of code
|
2020-08-07 12:44:36 +02:00 |
|
Alessia Bardi
|
b3f5f38c02
|
sparql query to get the records without mandatory fields
|
2020-08-07 12:31:41 +02:00 |
|
Alessia Bardi
|
f8a3bc5a0e
|
sparql query to get the collections without mandatory fields
|
2020-08-07 12:28:17 +02:00 |
|
Alessia Bardi
|
e82043d9eb
|
added comments
|
2020-08-07 11:45:09 +02:00 |
|
Alessia Bardi
|
7040ad741b
|
moved SPARQL query for ADS 270 in dedicated folder
|
2020-08-06 20:29:06 +02:00 |
|
Alessia Bardi
|
6365fa583d
|
sparql templates for the different fix and enrichment phases
|
2020-08-06 20:28:16 +02:00 |
|
Alessia Bardi
|
cf2f2e6f8b
|
organised templates and prepared folders for ADS SPARQL inserts
|
2020-08-06 20:26:01 +02:00 |
|
Enrico Ottonello
|
54458be7f5
|
order in sparql insert execution is important, this query contains all sparql insert in the correct order
|
2020-07-29 16:37:22 +02:00 |
|
Enrico Ottonello
|
220c6be6db
|
report returned to wf node
|
2020-07-24 13:15:27 +02:00 |
|
Enrico Ottonello
|
214502cc11
|
removed test publisher
|
2020-07-24 12:59:59 +02:00 |
|
Enrico Ottonello
|
f61f794c25
|
removed a few logs
|
2020-07-24 12:34:06 +02:00 |
|
Enrico Ottonello
|
11e00c806d
|
reset json parser status on each record
|
2020-07-24 01:48:17 +02:00 |
|
Enrico Ottonello
|
b29b4e1fd7
|
added log on json parser
|
2020-07-24 01:21:25 +02:00 |
|
Enrico Ottonello
|
eb61aebcd5
|
added other check on parsing json from rdf statements
|
2020-07-24 00:32:49 +02:00 |
|
Enrico Ottonello
|
5eaa655f01
|
added other check on es return code
|
2020-07-23 23:12:07 +02:00 |
|
Enrico Ottonello
|
173b0c289e
|
to avoid overload on graphdb heap it is better to slowing down query execution
|
2020-07-22 23:47:08 +02:00 |
|
Enrico Ottonello
|
5bb65a8854
|
added response code from es; added sparql insert for copying ariadne_subject value into collection for ads; fix sparql insert for copying collection data into records, collection id condition is needed
|
2020-07-22 23:13:47 +02:00 |
|
Enrico Ottonello
|
766f8199f2
|
removed empty values/structures from record/collection to es
|
2020-07-17 17:42:08 +02:00 |
|
Enrico Ottonello
|
d6aafdf97d
|
es _id only for collection type record must be numeric format, for now we prepend 10000 to originalId value
|
2020-07-13 11:03:21 +02:00 |
|
Enrico Ottonello
|
6a804f176b
|
removed extent and audience fields, empty values cause error on portal
|
2020-07-09 16:35:04 +02:00 |
|
Enrico Ottonello
|
31b7c61f62
|
Merge branch 'master' of https://code-repo.d4science.org/D-Net/AriadnePlus
|
2020-07-08 22:00:33 +02:00 |
|
Enrico Ottonello
|
9358f1eaf2
|
collection data are now indexed with all required fields by es mapping
|
2020-07-08 22:00:28 +02:00 |
|
Alessia Bardi
|
eb805ebf9b
|
test for checking structural equality in new and old records in elasticsearch
|
2020-07-08 16:34:48 +02:00 |
|
Alessia Bardi
|
7b644e04da
|
added dep for json diff
|
2020-07-08 16:34:03 +02:00 |
|
Alessia Bardi
|
4cf842f906
|
test for checking structural equality in new and old records in elasticsearch
|
2020-07-08 16:33:40 +02:00 |
|
Enrico Ottonello
|
364d25f2b7
|
archeologicalResourceType.id mapped
|
2020-07-08 11:54:32 +02:00 |
|
Enrico Ottonello
|
e689623994
|
all ES fields are now mapped
|
2020-07-08 10:50:14 +02:00 |
|
Enrico Ottonello
|
9737184b1e
|
start aligning es json fields to es mapping
|
2020-07-07 13:39:22 +02:00 |
|
Enrico Ottonello
|
db9b70feb0
|
new node and workflow to index on ES
|
2020-06-16 02:36:16 +02:00 |
|
Enrico Ottonello
|
42c3f5e885
|
added indexing of collection data with dedicated query
|
2020-06-15 12:10:59 +02:00 |
|
Enrico Ottonello
|
06995feefa
|
added rest method indexOnES(datasource,collectionId) to retrieve records data from graphDB and index to ES
|
2020-06-12 18:14:41 +02:00 |
|
Enrico Ottonello
|
064f298329
|
added query to retrieve record ids collection; datasource, collection id and record id are now paramters on a query template from file
|
2020-06-11 18:20:42 +02:00 |
|
Enrico Ottonello
|
b80274fff3
|
added graphDB reader and ES indexer
|
2020-06-10 19:39:53 +02:00 |
|
Enrico Ottonello
|
fa1d268b36
|
added query to link data from collection level to records level
|
2020-06-10 11:53:37 +02:00 |
|
Enrico Ottonello
|
af2eb7bb91
|
added sparql query folder
|
2020-06-09 11:44:48 +02:00 |
|
Enrico Ottonello
|
aa1c6c2fda
|
added query to generate aat and periodo enriched graphs
|
2020-06-09 11:39:56 +02:00 |
|
Enrico Ottonello
|
9814069fb2
|
new rest method to import data into graphDB from url
|
2020-05-30 17:21:03 +02:00 |
|
Enrico Ottonello
|
30b3fa2140
|
New JobNode and workflow to enrich content on GraphDB
|
2020-05-29 16:19:55 +02:00 |
|
Enrico Ottonello
|
739dcc6b82
|
added rest method to execute sparql insert query on graphdb
|
2020-05-27 23:07:08 +02:00 |
|
Enrico Ottonello
|
d23f08f750
|
removed BIND not necessary
|
2020-04-03 15:53:52 +02:00 |
|
Enrico Ottonello
|
a005328dfe
|
fix resource type variable name
|
2020-03-31 11:10:41 +02:00 |
|
Enrico Ottonello
|
d52ad32f68
|
added narrow and broad for aat match
|
2020-03-20 17:46:55 +01:00 |
|
Enrico Ottonello
|
7ae6d400b6
|
temporal coverage joined with periodO data that must be directly imported in graphDb in turtle format together with periodO mapping
|
2020-03-20 13:01:54 +01:00 |
|
Enrico Ottonello
|
ef42eb551a
|
fix spatial subselect: region, region point and region bounding box data must be referenced by 3 different variables inside the query
|
2020-03-17 16:27:14 +01:00 |
|
Enrico Ottonello
|
32b91c277d
|
all not madatory sets of conditions must be set as optional
|
2020-03-16 19:16:42 +01:00 |