michal.politowski
/
UrlsController
forked from lsmyrnaios/UrlsController
1
0
Fork 0
Code Pull Requests Activity
The Controller app of the PDF Aggregation Service.
25 Commits 3 Branches 14 Tags 1.1 MiB
Java 98.3%
Shell 1.6%
Go to file
Lampros Smyrnaios 780ed15ce2 - Fix a "databaseLock" bug, which could cause both the payload and attempt inserts and the "mergeParquetFiles" to fail, as the inserts could be executed concurrently with tables-compaction. 
- Fix the "null" representation of an "unknown" payload-size in the database.
- Remove the obsolete thread-locking for the "CreateDatabase" operation. This code is guaranteed to run BEFORE any other operation in the database.
- Implement the "handlePreparedStatementException" and "closeConnection" methods.
- Improve error-logs.
- Update dependencies.
- Code cleanup.
 		2021-11-30 13:26:19 +02:00
gradle/wrapper - Fix a "databaseLock" bug, which could cause both the payload and attempt inserts and the "mergeParquetFiles" to fail, as the inserts could be executed concurrently with tables-compaction. 2021-11-30 13:26:19 +02:00
scripts Initial commit of UrlsController. 2021-03-16 15:25:15 +02:00
src/main - Fix a "databaseLock" bug, which could cause both the payload and attempt inserts and the "mergeParquetFiles" to fail, as the inserts could be executed concurrently with tables-compaction. 2021-11-30 13:26:19 +02:00
README.md - Add the "installAndRun.sh" script. 2021-09-09 15:56:37 +03:00
build.gradle - Fix a "databaseLock" bug, which could cause both the payload and attempt inserts and the "mergeParquetFiles" to fail, as the inserts could be executed concurrently with tables-compaction. 2021-11-30 13:26:19 +02:00
installAndRun.sh - Fix a "databaseLock" bug, which could cause both the payload and attempt inserts and the "mergeParquetFiles" to fail, as the inserts could be executed concurrently with tables-compaction. 2021-11-30 13:26:19 +02:00
settings.gradle - Add the "isControllerAlive"-endpoint. 2021-09-23 15:08:52 +03:00

README.md

UrlsController

This is the Controller's Application.
It receives requests coming from the workers , constructs an assignments-list with data received from a database and returns the list to the workers.
Then it receives the "WorkerReports" and writes them into the database.
The database used is the Impala .
[...]

To install and run the application, run git clone and then execute the installAndRun.sh script.
If you want to just run the app, then run the script with the argument "1": ./installAndRun.sh 1.