Commit Graph

17 Commits

Author SHA1 Message Date
Lampros Smyrnaios 344bc46e08 Update Gradle. 2023-04-22 06:44:04 +03:00
Lampros Smyrnaios 0997558347 Update dependencies. 2023-04-20 15:39:15 +03:00
Lampros Smyrnaios 4da54e7a7d - Show a warning, in case the number of archived files is different from the number of requested files.
- Code polishing.
- Update Gradle.
2023-03-07 16:25:10 +02:00
Lampros Smyrnaios 9888349bef Update Gradle. 2023-02-20 19:14:16 +02:00
Lampros Smyrnaios 0626e85894 Update dependencies. 2023-02-15 16:18:33 +02:00
Lampros Smyrnaios 182d6153d4 - Set some optimization settings for gradle.
- Fix error-handling in "installAndRun.sh".
- Update dependencies.
2022-11-30 16:25:57 +02:00
Lampros Smyrnaios 01f12e2fe2 - Align with "PublicationsRetriever's" updated "couldRetry" and "wasValid" logic.
- Update dependencies.
2022-11-11 16:02:20 +02:00
Lampros Smyrnaios 31af0a81eb - Update the Worker's report to include the datasourceID for each record. It is used by the Controller inside the S3-fileNames.
- Update dependencies.
2022-04-01 19:42:32 +03:00
Lampros Smyrnaios 5fee05e994 Update dependencies. 2022-03-28 14:29:54 +03:00
Lampros Smyrnaios 3d1faf4a8a - Reduce memory-consumption in the long-run, by clearing some underlying data-structures after a threshold.
- Update Gradle.
2022-02-18 20:02:34 +02:00
Lampros Smyrnaios 1ddfd34236 - Allow the user to set a maximum number of assignments-batches for the Worker to handle. After handling those batches, the Worker will shut down. A number of < 0 > indicates an infinite number of batches.
- Avoid converting the zero fileSize to < null >. Now, the default value is < null >, so the zero-value will indicate a zero-byte file.
- Update dependencies.
- Code cleanup.
2021-12-24 00:12:34 +02:00
Lampros Smyrnaios 82d69f3bf5 - Calculate and set the max heap size with respect to the system resources, in "installAndRun.sh".
- Fix not setting the right "Error"-members when the docUrl was found, but the full-text was not retrieved.
- Set a "couldRetry"-indication in the "Error"-class, when the full-text was retrieved, as, in general, it could be retried to give the same successful result.
- Update the "docFileNotRetrieved"-check to use the standardized string.
- Eliminate some possible NPEs.
- Update Gradle.
2021-12-16 02:04:05 +02:00
Lampros Smyrnaios 018326eedd - Optimize the "FileZipper.zipMultipleFilesAndGetZip()" and "FileZipper.zipAFile()" methods.
- Improve the "getMultipleFullTexts"-endpoint. Check if the "fileNamesWithExtensions"-list is empty. Check if the baseDir for the fullTexts of a given assignments-counter is missing.
- Optimize the "PublicationsRetrieverPlugin.processAssignments()" method.
- Set a max-size limit to the amount of space the logs can use. Over that size, the older logs will be deleted.
- Show the heap size, in the beginning.
- Update Gradle.
- Code cleanup.
2021-12-03 04:09:40 +02:00
Lampros Smyrnaios 6355b3e397 - Increase the "PublicationsRetriever.threadsMultiplier" to "6", as the threads are mostly network-blocked.
- Make sure the "maven" package is installed before compiling the "PublicationsRetriever" library.
- Update dependencies.
2021-11-30 01:02:06 +02:00
Lampros Smyrnaios b6d66653f7 - Integrate the latest changes from the "PublicationsRetriever"-plugin.
- Update dependencies.
2021-09-01 19:42:32 +03:00
Lampros Smyrnaios 6cc2673fca - Add the ability to upload the files on an S3-ObjectStore.
- Change the server's port and the port of the controller-api.
- Update dependencies.
2021-07-29 09:01:53 +03:00
Lampros Smyrnaios 5f3409e072 - Update Spring and add the "gradle-wrapper.properties" file which defines the gradle version.
- Improve an info-logging message and cleanup the code.
2021-06-10 14:29:20 +03:00