Lampros Smyrnaios
dea257b87f
- Fix a bug, which caused the get-full-texts request to fail, because of the wrong "requestAssignmentsCounter".
...
- Fix a bug, which caused multiple workers to get assigned the same batch-counter, while the assignment-tasks where different.
- Set a max-size limit to the amount of space the logs can use. Over that size, the older logs will be deleted.
- Show the error-message returned from the Worker, when a getFullTexts-request fails.
- Improve some log-messages.
- Update dependencies.
- Code cleanup.
2021-12-06 20:18:30 +02:00
Lampros Smyrnaios
48eed20dd8
- Implement the "getAndUploadFullTexts" functionality. In order to access the S3-ObjectStore from one trusted place, the Controller will request the files from the workers and upload them on S3. Afterwards, the workers will delete those files from their local storage. Previously, each worker uploaded its own files.
...
- Move the "mergeParquetFiles" and "getCutBatchExceptionMessage" methods inside the "FileUtils" class.
- Code cleanup.
2021-11-30 18:23:27 +02:00
Lampros Smyrnaios
0d47c33a08
- Improve logging configurations.
...
- Fix the "AssignmentResponse.toString()" method.
- Update SpringBoot to v.2.5.6
- Code cleanup.
2021-11-04 11:57:19 +02:00
Lampros Smyrnaios
c194af167f
Allow handling of concurrent requests to the "getTestUrls"-endpoint.
2021-06-10 20:24:51 +03:00
Lampros Smyrnaios
308cab5ecd
- Return an HTTP-500-error when the server cannot find the resourceFile requested by the "getTestUrls"-endpoint.
...
- Close the "inputScanner" after each use when retrieving the test-tasks.
- Show info-logs when sending an assignment to a worker.
- Code cleanup.
2021-06-10 14:21:39 +03:00
Lampros Smyrnaios
787299b5b7
Add the "Datasource" inside the "Task" class and include it in the Assignment.
2021-05-20 02:50:50 +03:00
Lampros Smyrnaios
e2cc320baf
- Add the "getTestUrls"-endpoint which returns an "Assignment" with data retrieved from the added resource-file.
...
- Update the "getUrls"-endpoint to be ready to retrieve data from the database, once it's added.
- Update the dependencies.
- Code cleanup.
2021-05-18 17:23:20 +03:00
Lampros Smyrnaios