UrlsController/README.md

# UrlsController

This is the Controller's Application.<br>
It receives requests coming from the [workers](https://code-repo.d4science.org/lsmyrnaios/UrlsWorker) , constructs an assignments-list with data received from a database and returns the list to the workers.<br>
Then it receives the "WorkerReports" and writes them into the database.<br> 
The database used is the [Impala](https://impala.apache.org/) .<br>
[...] <br>

To install and run the application, run ```git clone```.
Then, provide a file *"S3_minIO_credentials.txt"*, inside the *working directory*.<br>
In the *"S3_minIO_credentials.txt"* file, you should provide the *endpoint*, the *accessKey*, the *secretKey*, the *region* and the *bucket*, in that order, separated by comma.
<br>
Afterwards, execute the ```installAndRun.sh``` script.<br>
If you want to just run the app, then run the script with the argument "1": ```./installAndRun.sh 1```.<br>
<br>
Initial commit of UrlsController. 2021-03-16 14:25:15 +01:00			`# UrlsController`

- Add the "installAndRun.sh" script. - Update the README. - Update the dependencies. 2021-09-09 14:56:37 +02:00			`This is the Controller's Application.<br>`
			`It receives requests coming from the [workers](https://code-repo.d4science.org/lsmyrnaios/UrlsWorker) , constructs an assignments-list with data received from a database and returns the list to the workers.<br>`
			`Then it receives the "WorkerReports" and writes them into the database.<br>`
			`The database used is the [Impala](https://impala.apache.org/) .<br>`
			`[...] <br>`

- Implement the "getAndUploadFullTexts" functionality. In order to access the S3-ObjectStore from one trusted place, the Controller will request the files from the workers and upload them on S3. Afterwards, the workers will delete those files from their local storage. Previously, each worker uploaded its own files. - Move the "mergeParquetFiles" and "getCutBatchExceptionMessage" methods inside the "FileUtils" class. - Code cleanup. 2021-11-30 17:23:27 +01:00			To install and run the application, run ```git clone```.
			`Then, provide a file "S3_minIO_credentials.txt", inside the working directory.<br>`
			`In the "S3_minIO_credentials.txt" file, you should provide the endpoint, the accessKey, the secretKey, the region and the bucket, in that order, separated by comma.`
			`<br>`
			Afterwards, execute the ```installAndRun.sh``` script.<br>
- Add the "installAndRun.sh" script. - Update the README. - Update the dependencies. 2021-09-09 14:56:37 +02:00			If you want to just run the app, then run the script with the argument "1": ```./installAndRun.sh 1```.<br>
			`<br>`