UrlsController/src/main/java/eu/openaire/urls_controller
Lampros Smyrnaios 88acaae20f - Replace the "numFullTextUrlsFound"-counter with "numFullTextsFound"-counter to reflect the end result of the actually available full-texts (which were downloaded by the Worker).
- Optimize the gather-fileNames loop.
- Improve a message in "installAndRun.sh"
2022-02-23 17:40:06 +02:00
..
components - Avoid getting and uploading (to S3), full-texts which are already uploaded by previous assignments-batches. 2021-12-21 15:55:27 +02:00
configuration - Add support for more than one full-text per id. Allow recognizing fileName additions: "id(1).pdf", "id(2).pdf", etc. 2022-02-07 13:57:09 +02:00
controllers - Improve performance when inserting records into the "attempt" table, by splitting the records equally, across more threads. 2022-02-22 13:54:16 +02:00
models Improve performance in the "getUrls"-endpoint, and more: 2021-11-30 19:59:46 +02:00
payloads - Fix a "databaseLock" bug, which could cause both the payload and attempt inserts and the "mergeParquetFiles" to fail, as the inserts could be executed concurrently with tables-compaction. 2021-11-30 13:26:19 +02:00
security - Add the "isControllerAlive"-endpoint. 2021-09-23 15:08:52 +03:00
util - Replace the "numFullTextUrlsFound"-counter with "numFullTextsFound"-counter to reflect the end result of the actually available full-texts (which were downloaded by the Worker). 2022-02-23 17:40:06 +02:00
Application.java - Improve performance when inserting records into the "attempt" table, by splitting the records equally, across more threads. 2022-02-22 13:54:16 +02:00