UrlsController/src/main/java/eu/openaire/urls_controller/util
Lampros Smyrnaios 0178e44574 - Increase security by sanitizing the value of the "workerId" before use it in sql-statements. Impala has bugs with some types of PreparedStatements.
- Improve reliability, by dropping the "current_assignment" table in case of an error, thus the next "getUrls"-request will not fail.
- Fix the "databaseLock" not being unlocked when the "addWorkerReport()" method returned early on some error-cases.
- Delete the "assignment"-data after inserting the related payloads and attempts in the database.
2021-12-10 21:47:58 +02:00
..
ControllerConstants.java - Implement the "getUrls" and "addWorkerReport" endpoints with full database-handling. 2021-11-09 23:59:27 +02:00
FileUnZipper.java - Implement the "getAndUploadFullTexts" functionality. In order to access the S3-ObjectStore from one trusted place, the Controller will request the files from the workers and upload them on S3. Afterwards, the workers will delete those files from their local storage. Previously, each worker uploaded its own files. 2021-11-30 18:23:27 +02:00
FileUtils.java - Increase security by sanitizing the value of the "workerId" before use it in sql-statements. Impala has bugs with some types of PreparedStatements. 2021-12-10 21:47:58 +02:00
GenericUtils.java Add the "Datasource" inside the "Task" class and include it in the Assignment. 2021-05-20 02:50:50 +03:00
S3ObjectStoreMinIO.java - Implement the "getAndUploadFullTexts" functionality. In order to access the S3-ObjectStore from one trusted place, the Controller will request the files from the workers and upload them on S3. Afterwards, the workers will delete those files from their local storage. Previously, each worker uploaded its own files. 2021-11-30 18:23:27 +02:00
UriBuilder.java - Increase the lower and upper limits for the Java Heap Size. 2021-12-06 20:27:39 +02:00