Commit Graph

93 Commits

Author SHA1 Message Date
Michele Artini c4fce785ab fixed a compilation problem of a unit test 2021-10-19 16:18:26 +02:00
Claudio Atzori 172363e7f1 [broker] integrating PR#147, notification record creation phase separated from indexing on ES 2021-10-19 15:56:27 +02:00
Claudio Atzori bdffa86c2f undo last commit 2021-10-19 15:39:38 +02:00
Michele Artini 210d6c0e6d generateNotificationsJob and indexNotificationsJob 2021-10-05 13:57:46 +02:00
Michele Artini 69008e20c2 log and tests 2021-10-05 11:58:20 +02:00
Michele Artini 8bbaa17335 reimplemented of conditions cache as a non static variable 2021-10-05 09:20:37 +02:00
Michele Artini 31a6ad1d79 optimization of verifySubsriptions() 2021-10-04 12:01:56 +02:00
Claudio Atzori ec94cc9b93 IndexNotificationsJob test: persist contents on HDFS instead of passing them to ES 2021-10-01 09:41:27 +02:00
Claudio Atzori 2ee21da43b suggestions from SonarLint 2021-08-11 12:13:22 +02:00
Claudio Atzori 908f57a475 code formatting 2021-07-29 10:49:39 +02:00
Claudio Atzori 4c5a71ba2f [broker] updated relation descriptors, making use of constant values 2021-07-28 17:11:18 +02:00
Claudio Atzori ae2b47b29d [broker] added coalesce(1) on the stats dataset before storing it on postgres 2021-07-09 15:47:51 +02:00
Claudio Atzori 5e4b91d9ef more pervasive use of constants from ModelConstants, especially for ORCID 2021-05-26 18:20:23 +02:00
Claudio Atzori 23b8883ab1 applied intellij code cleanup 2021-05-14 10:58:12 +02:00
Claudio Atzori 7941d7be29 WIP: using common definitions from ModelConstants 2021-03-31 18:33:57 +02:00
Michele Artini b9d90e95b8 Added eventId to ShortEventMessage 2021-01-14 14:32:31 +01:00
Michele Artini 64b0b0bfb3 fixed a bug with invalid subject topic 2021-01-14 14:32:31 +01:00
Michele Artini e3e0ab1de1 fixed a problem with join 2021-01-14 14:32:31 +01:00
Michele Artini 26a941315a openaireId 2021-01-14 14:32:31 +01:00
Michele Artini 6f4d1a37f0 ES wf properties 2021-01-14 14:32:31 +01:00
Michele Artini 3c9cbd19f3 whitelist of topics 2021-01-14 14:32:31 +01:00
Michele Artini 467aa77279 workingDir and outputDir 2021-01-14 14:32:31 +01:00
Michele Artini 10f3f7eca7 workingDir and outputDir 2021-01-14 14:32:31 +01:00
Michele Artini ff41a7b3a4 gzipped output 2021-01-14 14:32:31 +01:00
Claudio Atzori fba11eef2a cleanup 2020-12-07 21:53:13 +01:00
Michele Artini 5de8a7276f wf to partition opendoar events 2020-12-07 14:56:06 +01:00
Michele Artini ab08d12c46 considering abstract > MIN_LENGTH in ENRICH_MISSING_ABSTRACT 2020-11-19 10:42:10 +01:00
Michele Artini c96598aaa4 opendoar partition 2020-09-25 09:02:58 +02:00
Michele Artini 9e681609fd stats to sql file 2020-09-17 15:51:22 +02:00
Michele Artini 51321c2701 partition of events by opedoarId 2020-09-17 11:38:07 +02:00
Michele Artini 9b0c12f5d3 send notifications 2020-09-11 12:06:16 +02:00
Michele Artini 028613b751 remove old notifications 2020-09-09 15:32:06 +02:00
Michele Artini a597a218ab * forall topics 2020-09-08 16:39:40 +02:00
Michele Artini bb459caf69 support for all topic subscriptions 2020-08-27 11:01:21 +02:00
Michele Artini 82ed8edafd notification indexing 2020-08-26 15:10:48 +02:00
Michele Artini da470422d3 deleting events 2020-08-21 14:52:48 +02:00
Michele Artini 6e60bf026a indexing only a subset of eventsa 2020-08-19 12:39:22 +02:00
Michele Artini 8ba94833bd added an es prop 2020-07-29 14:16:08 +02:00
Michele Artini 346a1d2b5a update eventId generator 2020-07-18 09:40:36 +02:00
Michele Artini 262c29463e relations with multiple datasources 2020-07-15 09:18:40 +02:00
Michele Artini e1ae964bc4 stats 2020-07-10 16:12:08 +02:00
Michele Artini 2d742a84ae DedupConfig as json file 2020-07-09 12:53:46 +02:00
Michele Artini efadbdb2bc fixed a bug with duplicated events 2020-07-07 15:37:13 +02:00
Michele Artini 04bebb708c some fixes 2020-07-03 11:48:12 +02:00
Michele Artini b413db0bff white/blacklists 2020-07-02 12:43:03 +02:00
Michele Artini 3bcdfbabe9 list with limits 2020-07-01 08:42:39 +02:00
Michele Artini 59a5421c24 indexing, accumulators, limited lists 2020-06-30 16:17:09 +02:00
Michele Artini 6f13673464 accumulators 2020-06-29 16:33:32 +02:00
Michele Artini 35ae381d28 all events matchers 2020-06-29 08:43:56 +02:00
Michele Artini 2393d9da2f limits 2020-06-26 11:20:45 +02:00