From f89730f19628742d5e2ad94ef7d0a5b7b76a78e7 Mon Sep 17 00:00:00 2001 From: LSmyrnaios Date: Fri, 27 Jan 2023 14:31:07 +0200 Subject: [PATCH] Improve documentation. --- README.md | 9 +++++---- .../openaire/urls_controller/util/ParquetFileUtils.java | 2 +- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 5e9da05..0027fc1 100644 --- a/README.md +++ b/README.md @@ -4,15 +4,16 @@ The Controller's Application receives requests coming from the [Workers](https:/ Then, it receives the "WorkerReports", it requests the full-texts from the workers, in batches, and uploads them on the S3-Object-Store. Finally, it writes the related reports, along with the updated file-locations into the database.
The database used is the [Impala](https://impala.apache.org/).

-Statistics API: + +**Statistics API**: - "**getNumberOfPayloads**" endpoint: **http://IP:PORT/api/stats/getNumberOfPayloads** - "**getNumberOfRecordsInspected**" endpoint: **http://IP:PORT/api/stats/getNumberOfRecordsInspected**
-
-To install and run the application: + +**To install and run the application**: - Run ```git clone``` and then ```cd UrlsController```. -- Provide the **S3 Object Store** related configurations, inside the *src/main/resources/application.properties* file.
+- Set the preferable values inside the [__application.properties__](https://code-repo.d4science.org/lsmyrnaios/UrlsWorker/src/branch/master/src/main/resources/application.properties) file. - Execute the ```installAndRun.sh``` script which builds and runs the app.
If you want to just run the app, then run the script with the argument "1": ```./installAndRun.sh 1```.
If you want to build and run the app on a **Docker Container**, then run the script with the argument "0" followed by the argument "1": ```./installAndRun.sh 0 1```.
diff --git a/src/main/java/eu/openaire/urls_controller/util/ParquetFileUtils.java b/src/main/java/eu/openaire/urls_controller/util/ParquetFileUtils.java index d0e934f..1f77e95 100644 --- a/src/main/java/eu/openaire/urls_controller/util/ParquetFileUtils.java +++ b/src/main/java/eu/openaire/urls_controller/util/ParquetFileUtils.java @@ -402,7 +402,7 @@ public class ParquetFileUtils { // Write the parquet file. try ( InputStream inputS = Files.newInputStream(parquetFile.toPath()); OutputStream outS = conn.getOutputStream()) { int readByte = -1; while ( (readByte = inputS.read()) != -1 ) outS.write(readByte); - } + } // Any exception will be caught in the end of this method. statusCode = conn.getResponseCode(); if ( statusCode != 201 ) {