UrlsController/src/main/java/eu/openaire/urls_controller/services
Lampros Smyrnaios 8dfb58ee63 Avoid assigning the same publications multiple times to the Workers, after the recent "parallelization enchantment".
After that enchantment, each worker could request multiple assignment-batches, before its previous batches were processed by the Controller. This means that for each batch that was processed, the Controller was deleting from the "assignment" table, all the assignments (-batches) delivered to the Worker that brought that batch, even though the "attempt" and "payload" records for the rest of the batches were not inserted in the DB yet. So in a new assignments-batch request, the same publications that were already under processing, were delivered to the same or other Workers.
Now, for each finished batch, only the assignments of that batch are deleted from the "assignment" table.
2023-07-11 17:27:23 +03:00
..
BulkImportService.java - Show the workerIPs in the logs. 2023-05-29 12:12:08 +03:00
BulkImportServiceImpl.java Code polishing. 2023-05-29 12:21:48 +03:00
ShutdownService.java - Identify and handle a possible Worker-crash, in "UrlsServiceImpl.postReportResultToWorker()". 2023-06-15 23:19:36 +03:00
ShutdownServiceImpl.java - Identify and handle a possible Worker-crash, in "UrlsServiceImpl.postReportResultToWorker()". 2023-06-15 23:19:36 +03:00
StatsService.java Handle the case where the "stats"-queries are executed while some table of the DB are in a "merge" state. In this case, the queries fail and the Controller retries up to 10 times. 2023-07-06 18:29:13 +03:00
StatsServiceImpl.java Handle the case where the "stats"-queries are executed while some table of the DB are in a "merge" state. In this case, the queries fail and the Controller retries up to 10 times. 2023-07-06 18:29:13 +03:00
UrlsService.java - Process the WorkerReports in background Jobs and post the reportResults to the Workers. 2023-05-24 13:52:28 +03:00
UrlsServiceImpl.java Avoid assigning the same publications multiple times to the Workers, after the recent "parallelization enchantment". 2023-07-11 17:27:23 +03:00