UrlsWorker/src/main/java/eu/openaire/urls_worker
Lampros Smyrnaios 0ea7bccadb Leave the Max-Heap-Size to 8Gb, we assume that enough swap space will be available on the host.
We can still override the max-heap-size if desired.
2023-04-29 17:55:03 +03:00
..
components Use restTemplates with different read timeouts depending on the operation. For the assignments-request we need a shorter read timeout, than the one we need for the worker-report. This guarantees that the connection does not hungs for so long, when the Controller crashes before sending the assignments. 2023-04-29 17:24:16 +03:00
controllers Leave the Max-Heap-Size to 8Gb, we assume that enough swap space will be available on the host. 2023-04-29 17:55:03 +03:00
models Fix a "@JsonProperty" annotation inside "Payload.java". 2022-04-01 23:43:43 +03:00
payloads - Use the "Timestamp" data-type instead of the "Date", in order to include more information. 2021-11-27 02:37:33 +02:00
security - Add deletion, of the cookies in the newly-supported CookieManager, after each batch. 2022-06-27 17:58:02 +03:00
services - Improve performance of full-texts transferring to the Controller, by preloading some bytes for faster response to the Controller's read requests. 2023-03-08 13:12:17 +02:00
util Reduce memory usage. 2023-04-28 17:59:36 +03:00
UrlsWorkerApplication.java - Handle the case, where an instance of a urlReport record (having the same id and sourceUrl), may have failed to give a docUrl, due to en error, even if another instance gives the docUrl and the docFile. The absence of that handling could lead to a record-instance, being assigned a "fileLocation" which was actually an error-message (comment), and as a result the real "fileLocation" would have never been reached to be assigned, so the payload would be lost. 2023-02-21 15:22:49 +02:00