UrlsController/src/main/java/eu/openaire/urls_controller
Lampros Smyrnaios dea257b87f - Fix a bug, which caused the get-full-texts request to fail, because of the wrong "requestAssignmentsCounter".
- Fix a bug, which caused multiple workers to get assigned the same batch-counter, while the assignment-tasks where different.
- Set a max-size limit to the amount of space the logs can use. Over that size, the older logs will be deleted.
- Show the error-message returned from the Worker, when a getFullTexts-request fails.
- Improve some log-messages.
- Update dependencies.
- Code cleanup.
2021-12-06 20:18:30 +02:00
..
components - Implement the "getUrls" and "addWorkerReport" endpoints with full database-handling. 2021-11-09 23:59:27 +02:00
configuration - Fix a "databaseLock" bug, which could cause both the payload and attempt inserts and the "mergeParquetFiles" to fail, as the inserts could be executed concurrently with tables-compaction. 2021-11-30 13:26:19 +02:00
controllers - Fix a bug, which caused the get-full-texts request to fail, because of the wrong "requestAssignmentsCounter". 2021-12-06 20:18:30 +02:00
models Improve performance in the "getUrls"-endpoint, and more: 2021-11-30 19:59:46 +02:00
payloads - Fix a "databaseLock" bug, which could cause both the payload and attempt inserts and the "mergeParquetFiles" to fail, as the inserts could be executed concurrently with tables-compaction. 2021-11-30 13:26:19 +02:00
security - Add the "isControllerAlive"-endpoint. 2021-09-23 15:08:52 +03:00
util - Fix a bug, which caused the get-full-texts request to fail, because of the wrong "requestAssignmentsCounter". 2021-12-06 20:18:30 +02:00
Application.java - Implement the "getAndUploadFullTexts" functionality. In order to access the S3-ObjectStore from one trusted place, the Controller will request the files from the workers and upload them on S3. Afterwards, the workers will delete those files from their local storage. Previously, each worker uploaded its own files. 2021-11-30 18:23:27 +02:00