forked from lsmyrnaios/UrlsController
- Add documentation about the "BulkImport API" in the README.
- Fix a link in README. - Update dependencies.
This commit is contained in:
parent
74ff31fc64
commit
03bf4294b8
10
README.md
10
README.md
|
@ -8,6 +8,14 @@ It can also process **Bulk-Import** requests, from compatible data sources, in w
|
||||||
For interacting with the database we use [Impala](https://impala.apache.org/).<br>
|
For interacting with the database we use [Impala](https://impala.apache.org/).<br>
|
||||||
<br>
|
<br>
|
||||||
|
|
||||||
|
**BulkImport API**:
|
||||||
|
- "**bulkImportFullTexts**" endpoint: **http://\<IP\>:<PORT>/api/bulkImportFullTexts?provenance=\<provenance\>&bulkImportDir=\<bulkImportDir\>&shouldDeleteFilesOnFinish={true|false}** <br>
|
||||||
|
This endpoint loads the right configuration with the help of the "provenance" param and then processes the full-texts files inside the given directory, in the following way: it generates the openAireIDs, uploads the files to the S3 ObjectStore, generates and stores the the "payload" records in the database. If it is specified, it removes the successfully imported full-texts from the directory.
|
||||||
|
- "**getBulkImportReport**" endpoint: **http://\<IP\>:<PORT>/api/getBulkImportReport?id=\<bulkImportReportId\>** <br>
|
||||||
|
This endpoint returns the bulkImport report, which corresponds to the given ID, in JSON format.
|
||||||
|
<br>
|
||||||
|
<br>
|
||||||
|
|
||||||
**Statistics API**:
|
**Statistics API**:
|
||||||
- "**getNumberOfAllPayloads**" endpoint: **http://\<IP\>:<PORT>/api/stats/getNumberOfAllPayloads** <br>
|
- "**getNumberOfAllPayloads**" endpoint: **http://\<IP\>:<PORT>/api/stats/getNumberOfAllPayloads** <br>
|
||||||
This endpoint returns the total number of payloads existing in the database, independently of the way they were aggregated. This includes the payloads created by other pieces of software, before the PDF-Aggregation-Service was created.
|
This endpoint returns the total number of payloads existing in the database, independently of the way they were aggregated. This includes the payloads created by other pieces of software, before the PDF-Aggregation-Service was created.
|
||||||
|
@ -22,7 +30,7 @@ For interacting with the database we use [Impala](https://impala.apache.org/).<b
|
||||||
|
|
||||||
**To install and run the application**:
|
**To install and run the application**:
|
||||||
- Run ```git clone``` and then ```cd UrlsController```.
|
- Run ```git clone``` and then ```cd UrlsController```.
|
||||||
- Set the preferable values inside the [__application.properties__](https://code-repo.d4science.org/lsmyrnaios/UrlsWorker/src/branch/master/src/main/resources/application.properties) file.
|
- Set the preferable values inside the [__application.yml__](https://code-repo.d4science.org/lsmyrnaios/UrlsController/src/branch/master/src/main/resources/application.yml) file.
|
||||||
- Execute the ```installAndRun.sh``` script which builds and runs the app.<br>
|
- Execute the ```installAndRun.sh``` script which builds and runs the app.<br>
|
||||||
If you want to just run the app, then run the script with the argument "1": ```./installAndRun.sh 1```.<br>
|
If you want to just run the app, then run the script with the argument "1": ```./installAndRun.sh 1```.<br>
|
||||||
If you want to build and run the app on a **Docker Container**, then run the script with the argument "0" followed by the argument "1": ```./installAndRun.sh 0 1```.<br>
|
If you want to build and run the app on a **Docker Container**, then run the script with the argument "0" followed by the argument "1": ```./installAndRun.sh 0 1```.<br>
|
||||||
|
|
|
@ -40,10 +40,10 @@ dependencies {
|
||||||
// Enable the validation annotations.
|
// Enable the validation annotations.
|
||||||
//implementation group: 'jakarta.validation', name: 'jakarta.validation-api', version: '3.0.2'
|
//implementation group: 'jakarta.validation', name: 'jakarta.validation-api', version: '3.0.2'
|
||||||
|
|
||||||
implementation "org.projectlombok:lombok:1.18.26"
|
implementation "org.projectlombok:lombok:1.18.28"
|
||||||
|
|
||||||
// https://mvnrepository.com/artifact/com.google.guava/guava
|
// https://mvnrepository.com/artifact/com.google.guava/guava
|
||||||
implementation group: 'com.google.guava', name: 'guava', version: '31.1-jre'
|
implementation group: 'com.google.guava', name: 'guava', version: '32.0.0-jre'
|
||||||
|
|
||||||
// https://mvnrepository.com/artifact/org.apache.commons/commons-lang3
|
// https://mvnrepository.com/artifact/org.apache.commons/commons-lang3
|
||||||
implementation group: 'org.apache.commons', name: 'commons-lang3', version: '3.12.0'
|
implementation group: 'org.apache.commons', name: 'commons-lang3', version: '3.12.0'
|
||||||
|
|
Loading…
Reference in New Issue