forked from lsmyrnaios/UrlsController
Improve documentation.
This commit is contained in:
parent
dc8f0f2bd1
commit
f89730f196
|
@ -4,15 +4,16 @@ The Controller's Application receives requests coming from the [Workers](https:/
|
|||
Then, it receives the "WorkerReports", it requests the full-texts from the workers, in batches, and uploads them on the S3-Object-Store. Finally, it writes the related reports, along with the updated file-locations into the database.<br>
|
||||
The database used is the [Impala](https://impala.apache.org/).<br>
|
||||
<br>
|
||||
Statistics API:
|
||||
|
||||
**Statistics API**:
|
||||
- "**getNumberOfPayloads**" endpoint: **http://IP:PORT/api/stats/getNumberOfPayloads**
|
||||
- "**getNumberOfRecordsInspected**" endpoint: **http://IP:PORT/api/stats/getNumberOfRecordsInspected**
|
||||
<br>
|
||||
|
||||
<br>
|
||||
To install and run the application:
|
||||
|
||||
**To install and run the application**:
|
||||
- Run ```git clone``` and then ```cd UrlsController```.
|
||||
- Provide the **S3 Object Store** related configurations, inside the *src/main/resources/application.properties* file.<br>
|
||||
- Set the preferable values inside the [__application.properties__](https://code-repo.d4science.org/lsmyrnaios/UrlsWorker/src/branch/master/src/main/resources/application.properties) file.
|
||||
- Execute the ```installAndRun.sh``` script which builds and runs the app.<br>
|
||||
If you want to just run the app, then run the script with the argument "1": ```./installAndRun.sh 1```.<br>
|
||||
If you want to build and run the app on a **Docker Container**, then run the script with the argument "0" followed by the argument "1": ```./installAndRun.sh 0 1```.<br>
|
||||
|
|
|
@ -402,7 +402,7 @@ public class ParquetFileUtils {
|
|||
// Write the parquet file.
|
||||
try ( InputStream inputS = Files.newInputStream(parquetFile.toPath()); OutputStream outS = conn.getOutputStream()) {
|
||||
int readByte = -1; while ( (readByte = inputS.read()) != -1 ) outS.write(readByte);
|
||||
}
|
||||
} // Any exception will be caught in the end of this method.
|
||||
|
||||
statusCode = conn.getResponseCode();
|
||||
if ( statusCode != 201 ) {
|
||||
|
|
Loading…
Reference in New Issue