UrlsWorker/src/main/java/eu/openaire/urls_worker
Lampros Smyrnaios 2aedae2367 - In case a serious error happened while processing the assignments, instead of shutting down immediately, now the Worker shuts down the executor service, registers that it will shut down soon and waits for the Controller to retrieve the already downloaded full-text files.
- In case the full-texts' subdirectory could not be created, then terminate the "handleAssignment" method immediately. No posting of a faulty workerReport to the Controller should happen.
- Code polishing.
2023-05-31 15:25:36 +03:00
..
components - In case a serious error happened while processing the assignments, instead of shutting down immediately, now the Worker shuts down the executor service, registers that it will shut down soon and waits for the Controller to retrieve the already downloaded full-text files. 2023-05-31 15:25:36 +03:00
controllers - In case a serious error happened while processing the assignments, instead of shutting down immediately, now the Worker shuts down the executor service, registers that it will shut down soon and waits for the Controller to retrieve the already downloaded full-text files. 2023-05-31 15:25:36 +03:00
models Fix a "@JsonProperty" annotation inside "Payload.java". 2022-04-01 23:43:43 +03:00
payloads - After a WorkerReport has been sent, ask for new assignments immediately. So, the Worker does not have to wait for hours for the Controller to check for duplicate files in the DB, retrieve and upload the full-texts and insert the records to the DB. 2023-05-23 22:19:41 +03:00
security - Add deletion, of the cookies in the newly-supported CookieManager, after each batch. 2022-06-27 17:58:02 +03:00
services - Improve performance of full-texts transferring to the Controller, by preloading some bytes for faster response to the Controller's read requests. 2023-03-08 13:12:17 +02:00
util - After a WorkerReport has been sent, ask for new assignments immediately. So, the Worker does not have to wait for hours for the Controller to check for duplicate files in the DB, retrieve and upload the full-texts and insert the records to the DB. 2023-05-23 22:19:41 +03:00
UrlsWorkerApplication.java - After a WorkerReport has been sent, ask for new assignments immediately. So, the Worker does not have to wait for hours for the Controller to check for duplicate files in the DB, retrieve and upload the full-texts and insert the records to the DB. 2023-05-23 22:19:41 +03:00