21 lines
1.7 KiB
Markdown
21 lines
1.7 KiB
Markdown
# UrlsWorker
|
|
|
|
The Worker's Application, requests assignments from the [Controller](https://code-repo.d4science.org/lsmyrnaios/UrlsController) and processes them, downloading the available full-texts.<br>
|
|
Then, it posts the results to the Controller, which in turn, requests from the Worker, the full-texts which are not already found by other workers, in batches.<br>
|
|
The Worker responds by compressing and sending the requested files in each batch.<br>
|
|
<br>
|
|
To install and run the application:
|
|
- Run ```git clone``` and then ```cd UrlsWorker```.
|
|
- [Optional] Create the file ```inputData.txt``` , which contains just one line with the ___workerId___, the ___maxAssignmentsLimitPerBatch___, the ___maxAssignmentsBatchesToHandleBeforeRestart___, the ___controller's base api-url___ and the ___shutdownOrCancelCode___, all seperated by a _comma_ "```,```" .<br>
|
|
For example: ```worker_1,1000,0,http://IP:PORT/api/,stopOrCancelCode```.<br>
|
|
The ___shutdownOrCancelCode___ is kind of an "auth-code", when receiving "__shutdown__" and "__cancel-shutdown__" requests.
|
|
- Execute the ```installAndRun.sh``` script.<br>
|
|
In case the above file (_inputData.txt_) does not exist, the script will request the required data from the user, and then it will create the _inputData.txt_ file.<br>
|
|
<br>
|
|
|
|
Notes:
|
|
- If the "maxAssignmentsBatchesToHandleBeforeRestart" is zero or negative, then an infinite number of assignments-batches will be handled.<br>
|
|
- The above script, installs the [PublicationsRetriever](https://github.com/LSmyrnaios/PublicationsRetriever), as a library and then compiles and runs the whole Application.<br>
|
|
- If you want to just run the app, then run the script with the argument "1": ```./installAndRun.sh 1```.<br>
|
|
<br>
|