# UrlsWorker

The Worker's Application, requests assignments from the [Controller](https://code-repo.d4science.org/lsmyrnaios/UrlsController) and processes them with the help of the [__PublicationsRetriever__](https://github.com/LSmyrnaios/PublicationsRetriever) software and downloads the available full-texts.<br>
Then, it posts the results to the Controller, which in turn, requests from the Worker, the full-texts which are not already found by other workers, in batches.<br>
The Worker responds by compressing and sending the requested files, in each batch.<br>
<br>
We use Facebook's [**Zstandard**](https://facebook.github.io/zstd/) compression algorithm, which brings very big benefits in compression rate and speed.  
<br>
<br>

**To install and run the application**:
- Run ```git clone``` and then ```cd UrlsWorker```.
- Set the preferable values inside the [__application.properties__](https://code-repo.d4science.org/lsmyrnaios/UrlsWorker/src/branch/master/src/main/resources/application.properties) file.
- Execute the ```installAndRun.sh``` script.<br>
<br>

**Notes**:
- The above script, installs the [PublicationsRetriever](https://github.com/LSmyrnaios/PublicationsRetriever) software, as a library and then compiles and runs the whole Application.<br>
- If you want to just run the app, then run the script with the argument "1": ```./installAndRun.sh 1```. In this scenario, apart from the SpringBoot-app not re-building,
the [PublicationsRetriever](https://github.com/LSmyrnaios/PublicationsRetriever) software, will not be re-installed as well.<br>
- If you want to avoid re-installing the [PublicationsRetriever](https://github.com/LSmyrnaios/PublicationsRetriever) software, but still building the Worker's application, run the script with the argument "0", followed by the argument "1": ```./installAndRun.sh 0 1```, i.e. when using a development (non-published) version
or when nothing has changed and wanting to avoid the time-overhead.<br>
<br>