Lampros Smyrnaios lsmyrnaios
  • Joined on Apr 01, 2019
Loading Heatmap…

lsmyrnaios pushed to master at lsmyrnaios/UrlsController

  • a01e11eef0 When all the data is processed, increase the number of "max-attempts" to retry some very old records, in the next requests.

4 hours ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsController

  • 3c9f8870d1 - Change the repository for the Impala JDBC Driver, as the previous one had networking issues.

9 hours ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsController

  • ff46839158 Fix not prioritizing the gradle version defined inside the "installAndRun.sh" script.

6 days ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsWorker

  • a428b1d1e6 - Fix not prioritizing the gradle version defined inside the "installAndRun.sh" script.

6 days ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsController

6 days ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsController

  • ab99bc6168 - Make sure the temp table "current_assignment" from a cancelled previous execution, is dropped and purged on startup.

1 week ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsWorker

  • 8912bb1cf9 Fix adding an invalid error-message in case of an "alreadyDownloaded" full-text being discovered inside the "FileUtils.dataToBeLoggedList".

1 week ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsWorker

  • 0032a8018f - Improve search-accuracy of "alreadyDownloaded" full-texts.

1 week ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsWorker

  • d61ff4b6dd Integrate some changes from the "PublicationsRetrieverPlugin".

2 weeks ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsController

  • 83f40a23d9 Bring back the prepared-statements for the insert-queries. After the fix of the "broken pipe"-error, they now work. Bringing them back, increases security and solves the "SQL syntax errors" caused by the values of some URLs.
  • 2cf25b0d26 - Fix Impala "broken pipe" error, by closing the connection when not in need. The connection is reopened later with minimal overhead, as a connection pool is used.
  • Compare 2 commits »

2 weeks ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsWorker

  • 8abb260d60 - In case of an unknown (non-documented) exception inside "LoaderAndChecker.invokeAllTasksAndWait", now it will be logged and the app will gently shut down with an error-message in the Error-stream.

3 weeks ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsWorker

  • 92d011e8a0 - Make sure the handled assignments - full-texts are deleted before the application exits.

4 weeks ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsController

  • 82bf11b9b3 - Workaround a bug of Impala-JDBC-Driver, when creating insert-prepared-statements.

1 month ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsWorker

  • 1ddfd34236 - Allow the user to set a maximum number of assignments-batches for the Worker to handle. After handling those batches, the Worker will shut down. A number of < 0 > indicates an infinite number of batches.

1 month ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsController

  • 33ba3e8d91 - Avoid getting and uploading (to S3), full-texts which are already uploaded by previous assignments-batches.

1 month ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsWorker

  • a8e2ddcf54 - Reduce the "PublicationsRetriever.threadsMultiplier" to 4.

1 month ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsWorker

  • c46c8c448a - Upgrade the zip-file delivery by using the "InputStreamResource". This way is more reliable, have better performance and uses less memory.

1 month ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsWorker

  • 4fb5becace - Increase the system-reserved memory, in "installAndRun.sh".

1 month ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsWorker

  • 82d69f3bf5 - Calculate and set the max heap size with respect to the system resources, in "installAndRun.sh".

1 month ago

lsmyrnaios pushed to master at lsmyrnaios/UrlsWorker

  • 0db35a83e7 - Reduce memory consumption and fix a potential issue, where many "already-retrieved" full-texts would be already deleted (or in different directories, for a short time), as they belonged to a previous assignments-batch (this case is now possible, after the following fix).

1 month ago