Commit Graph

30 Commits

Author SHA1 Message Date
Enrico Ottonello 5a68ab07b9 mapping loaded only once now 2021-01-29 17:16:48 +01:00
Enrico Ottonello b8d0e3f741 enrichment queries set is now splitted on workflow node side, because of timeout error on very large collection 2021-01-07 23:40:47 +01:00
Enrico Ottonello 5d29077a43 added counter on wf progress information 2020-10-20 19:34:40 +02:00
Enrico Ottonello 8073ac7131 the node retrieves resources identifier list for collection and record, then loops on the 2 identifiers lists and request indexing to rest module 2020-10-20 17:43:27 +02:00
Enrico Ottonello 0e96774895 changed http request method for indexing 2020-10-15 17:09:35 +02:00
Enrico Ottonello 25de647504 set KeepAlive true on used socket; replaced SimpleJobNode with AsyncJobNode 2020-10-13 15:34:43 +02:00
Enrico Ottonello 2e732a3b5b sparql insert for ads 1;replaced AsynchJobNode with SimpleJobNode to 404 problem 2020-10-10 18:05:02 +02:00
Enrico Ottonello 8b3f8896a5 removed progress bar handler attached to resultset client (not used here) 2020-10-10 16:53:37 +02:00
Enrico Ottonello db2ad3f97e multithreads http client not needed on indexjobnode, maybe avoid 404 response after a few minutes, added log on each indexing operation 2020-08-12 15:47:55 +02:00
Enrico Ottonello 5c3ef5f25b retrieved body from server response 2020-07-24 18:32:28 +02:00
Enrico Ottonello e189283059 retrieved report on index results 2020-07-24 17:59:23 +02:00
Enrico Ottonello db9b70feb0 new node and workflow to index on ES 2020-06-16 02:36:16 +02:00
Alessia Bardi 4d68ecad58 the job fails if the endpoint returns client errors 4xx 2020-06-05 16:19:54 +02:00
Enrico Ottonello 819ef88520 new workflow to import periodo from url 2020-06-01 12:29:48 +02:00
Enrico Ottonello 30b3fa2140 New JobNode and workflow to enrich content on GraphDB 2020-05-29 16:19:55 +02:00
Enrico Ottonello 226cdde77d all records related to an api are published into the same graph 2020-02-19 14:33:54 +01:00
Enrico Ottonello 17552bafa0 foreach published api, the datasource name is stored into graph datasourceApis 2020-01-24 10:52:04 +01:00
Enrico Ottonello 8530e488d0 all records related to a datasource api are now published on graphdb into n named graphs because of out of memory issue 2020-01-14 16:55:45 +01:00
Enrico Ottonello e748d3d802 added progress status data on publishToGraphDB workflow node ui 2019-12-18 15:55:53 +01:00
Enrico Ottonello 7980449ac0 CloseableHttpClient instance has to be closed after all post has been executed, same handling for PoolingHttpClientConnectionManager instance shutdown 2019-12-17 15:35:36 +01:00
Alessia Bardi 11e3992ab4 slow down the task producer to avoid OOM errors. See here for more: https://stackoverflow.com/questions/42108351/executorservice-giving-out-of-memory-error 2019-12-16 18:52:08 +01:00
Alessia Bardi c770fc40e0 cleaning up: let's remove the classes we do not need anymore, like Virtuoso-related classes and specific plugins that we have created for Parthenos 2019-12-16 18:34:08 +01:00
Enrico Ottonello 5087750f96 rdf handling move away from this node 2019-12-16 15:32:20 +01:00
Enrico Ottonello 98b452c59a new node to elastic search index 2019-12-16 15:25:55 +01:00
Enrico Ottonello 0f6f2e7b75 after records publishing the datasourceApi informations are saved on GraphDB 2019-12-13 14:55:51 +01:00
Enrico Ottonello 2f5fb6fcb5 new module for graphdb publishing 2019-12-12 12:58:30 +01:00
Enrico Ottonello b3a8f05c26 x3m mapping is now only taken from url; ads policy aligned to aggregator instance; removed wf parameter unused 2019-10-31 14:08:37 +01:00
Enrico Ottonello e968c5faa5 mapping url downloaded from remote url set in workflow parameters 2019-10-22 13:45:17 +02:00
Enrico Ottonello 05209e8393 added custom plugin to ads datasource 2019-09-19 15:09:44 +02:00
Enrico Ottonello a9f6a0f6eb added modules related to ariadneplus 2019-06-18 16:15:23 +02:00