Commit Graph

101 Commits

Author SHA1 Message Date
Alessia Bardi 6418d27d32 removed '<>' in wf param, otherwise we get an exception when the workflow is assigned 2023-03-27 11:34:40 +02:00
Alessia Bardi 354c8e50fd Updated mail template for wf notifications of ariadne 2023-01-25 16:25:31 +01:00
Alessia Bardi d9ea174948 better default values for wf template of periodo 2022-10-10 17:06:53 +02:00
Alessia Bardi fa71f9a7e1 Incremental support for thanados plugin 2022-05-05 16:13:32 +02:00
Alessia Bardi ae2b4b6388 protovol and plugin bean for thanados 2022-05-05 15:44:04 +02:00
Alessia Bardi fcdec764f1 Fixed end iteration criteria 2022-04-19 17:03:54 +02:00
Alessia Bardi 48b135190e Added some printlns while testing 2022-04-15 14:42:59 +02:00
Alessia Bardi 305bc617ff Collector plugin for Thanados 2022-04-11 18:55:54 +02:00
Enrico Ottonello 62af1fc637 update to ARIADNEplus Generator Policy version 1.10 2022-03-30 10:53:00 +02:00
Enrico Ottonello f8e6e04e82 added charset on payload creation for nara publishing 2022-01-31 15:25:38 +01:00
Enrico Ottonello 98cccc4442 removed GZIPCompression on request, currently it is not needed 2021-12-17 11:44:16 +01:00
Enrico Ottonello a90f02487a record to publish now are sent to spring module inside body request, because of htmlsimple harvesting can produce large record (>10MB) 2021-12-17 11:39:37 +01:00
Alessia Bardi 0d0ebc4d77 testing DIME 2021-10-25 15:24:23 +02:00
Enrico Ottonello 788fc3b734 added lowercase function for derivedSubject.prefLabe, nativeSubject.prefLabel 2021-10-19 16:03:38 +02:00
Enrico Ottonello e9430ab460 changed title structure from field to object for multiple language handling 2021-10-18 18:06:40 +02:00
Enrico Ottonello 3510d7d1be Merge branch 'new_es_mapping'
# Conflicts:
#	dnet-ariadneplus-graphdb-publisher/src/main/resources/eu/dnetlib/ariadneplus/sparql/read_collection_data_template.sparql
#	dnet-ariadneplus-graphdb-publisher/src/main/resources/eu/dnetlib/ariadneplus/sparql/read_record_data_template.sparql
#	dnet-ariadneplus-graphdb-publisher/test/java/eu/dnetlib/ariadneplus/GraphDbReaderAndESIndexTest.java
2021-09-08 15:56:12 +02:00
Enrico Ottonello 03da34a897 added dynamic property set to false to avoid es mapping modification indexinga record; previous index has been deleted 2021-06-15 23:38:02 +02:00
Alessia Bardi fc71bc24ed new generator policy 2021-06-04 10:20:46 +02:00
Enrico Ottonello 06f20f7a8d shell command to create the new elastic search index with mapping on staging 2021-05-31 16:59:56 +02:00
Enrico Ottonello 5a68ab07b9 mapping loaded only once now 2021-01-29 17:16:48 +01:00
Enrico Ottonello 290259fbc2 new collector plugin (xmlsFolder) based on ads one, adding a new parameter which represents the list of namespaces that has to be added to the generated xml record during collection step 2021-01-15 13:33:23 +01:00
Miriam Baglioni 472ee2754c Aggiornare 'dnet-ariadneplus/src/main/resources/es/port.py' 2021-01-08 16:52:26 +01:00
Miriam Baglioni 060f7f8673 Aggiornare 'dnet-ariadneplus/src/main/resources/es/port.py' 2021-01-08 16:51:44 +01:00
Miriam Baglioni 538be9e035 Aggiungi 'dnet-ariadneplus/src/main/resources/es/port.py' 2021-01-08 16:51:07 +01:00
Enrico Ottonello b8d0e3f741 enrichment queries set is now splitted on workflow node side, because of timeout error on very large collection 2021-01-07 23:40:47 +01:00
Enrico Ottonello 4673c143ce curl command to create catalog index with mapping for elastic search on production 2021-01-07 11:48:00 +01:00
Enrico Ottonello 1ca29a2ebf alignment with current elastic-test instance mapping 2021-01-07 11:35:33 +01:00
Enrico Ottonello a05d8e97da replicated with a new id for having different execution dates with aggregation_to_graphdb_wf.xml 2020-12-18 17:39:15 +01:00
Enrico Ottonello 3e68138b81 new wf to publish a collection on staging graphdb and then on public one 2020-12-18 16:20:38 +01:00
Alessia Bardi 5f695ee538 fixed worng copy&paste, probably from an st file...again 2020-12-17 20:15:51 +01:00
Alessia Bardi 0e30eccbae fixed worng copy&paste, probably from an st file 2020-12-17 17:56:13 +01:00
Alessia Bardi 6c150be5b2 added missing param 2020-12-17 13:53:26 +01:00
Alessia Bardi e5eefef411 fixed param name 2020-12-16 16:09:40 +01:00
Alessia Bardi fa3592b1b3 added missing wf node 2020-12-15 17:06:52 +01:00
Alessia Bardi 687d5ab9ec only one start node 2020-12-14 18:27:23 +01:00
Alessia Bardi be64ce5dcf changed workflow name ad description 2020-12-14 18:22:13 +01:00
Alessia Bardi 8a9782cb1e re-organising workflows and templates for staging/public instances of graphdb and elasticsearch 2020-12-14 17:49:53 +01:00
Enrico Ottonello 5d29077a43 added counter on wf progress information 2020-10-20 19:34:40 +02:00
Enrico Ottonello 8073ac7131 the node retrieves resources identifier list for collection and record, then loops on the 2 identifiers lists and request indexing to rest module 2020-10-20 17:43:27 +02:00
Enrico Ottonello 0e96774895 changed http request method for indexing 2020-10-15 17:09:35 +02:00
Enrico Ottonello 25de647504 set KeepAlive true on used socket; replaced SimpleJobNode with AsyncJobNode 2020-10-13 15:34:43 +02:00
Enrico Ottonello 2e732a3b5b sparql insert for ads 1;replaced AsynchJobNode with SimpleJobNode to 404 problem 2020-10-10 18:05:02 +02:00
Enrico Ottonello 8b3f8896a5 removed progress bar handler attached to resultset client (not used here) 2020-10-10 16:53:37 +02:00
Enrico Ottonello db2ad3f97e multithreads http client not needed on indexjobnode, maybe avoid 404 response after a few minutes, added log on each indexing operation 2020-08-12 15:47:55 +02:00
Enrico Ottonello 5c3ef5f25b retrieved body from server response 2020-07-24 18:32:28 +02:00
Enrico Ottonello e189283059 retrieved report on index results 2020-07-24 17:59:23 +02:00
Alessia Bardi 74d0f440e3 Merge branch 'master' of https://code-repo.d4science.org/D-Net/AriadnePlus 2020-06-23 10:32:11 +02:00
Alessia Bardi e5d1f845a3 updated policy 2020-06-23 10:32:04 +02:00
Enrico Ottonello db9b70feb0 new node and workflow to index on ES 2020-06-16 02:36:16 +02:00
Alessia Bardi 4d68ecad58 the job fails if the endpoint returns client errors 4xx 2020-06-05 16:19:54 +02:00