UrlsWorker/src/main/java/eu/openaire/urls_worker
Lampros Smyrnaios 82d69f3bf5 - Calculate and set the max heap size with respect to the system resources, in "installAndRun.sh".
- Fix not setting the right "Error"-members when the docUrl was found, but the full-text was not retrieved.
- Set a "couldRetry"-indication in the "Error"-class, when the full-text was retrieved, as, in general, it could be retried to give the same successful result.
- Update the "docFileNotRetrieved"-check to use the standardized string.
- Eliminate some possible NPEs.
- Update Gradle.
2021-12-16 02:04:05 +02:00
..
components - Use the "Timestamp" data-type instead of the "Date", in order to include more information. 2021-11-27 02:37:33 +02:00
controllers - Improve Zip-file delivery and significantly decrease memory consumption, by streaming it, instead of loading the whole file in memory before sending it to the Controller. 2021-12-13 15:29:03 +02:00
exceptions - The worker will store the files in its local file-system and will send them to the controller in batches, after the latter requests them. When all files from a given assignments-num are sent, the files will be deleted from the Worker, in a scheduled-job. 2021-11-26 17:04:31 +02:00
models - Use the "Timestamp" data-type instead of the "Date", in order to include more information. 2021-11-27 02:37:33 +02:00
payloads - Use the "Timestamp" data-type instead of the "Date", in order to include more information. 2021-11-27 02:37:33 +02:00
plugins - Calculate and set the max heap size with respect to the system resources, in "installAndRun.sh". 2021-12-16 02:04:05 +02:00
security Initial commit of UrlsWorker. 2021-03-16 18:38:53 +02:00
services - Improve Zip-file delivery and significantly decrease memory consumption, by streaming it, instead of loading the whole file in memory before sending it to the Controller. 2021-12-13 15:29:03 +02:00
util - Improve Zip-file delivery and significantly decrease memory consumption, by streaming it, instead of loading the whole file in memory before sending it to the Controller. 2021-12-13 15:29:03 +02:00
UrlsWorkerApplication.java - Fix a bug, causing the "fileLocation" to be set to the value of the "errorCause", when the docFile was not retrieved. 2021-12-07 19:33:10 +02:00