lsmyrnaios
/
UrlsController
1
0
Fork 2
Code Issues Pull Requests Releases 18 Wiki Activity

Update README.md

This commit is contained in:
Lampros Smyrnaios 2024-09-19 13:02:11 +02:00
parent b8b83e3d74
commit 3f1e96e9f3
1 changed files with 4 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
README.md
View File

@ -23,7 +23,7 @@ For managing and generating data, we use [**Impala**](https://impala.apache.org/
**BulkImport API**: **BulkImport API**:
- "**bulkImportFullTexts**" endpoint: **http://\<IP\>:\<PORT\>/api/bulkImportFullTexts?provenance=\<provenance\>&bulkImportDir=\<bulkImportDir\>&shouldDeleteFilesOnFinish={true|false}** <br> - "**bulkImportFullTexts**" endpoint: **http://\<IP\>:\<PORT\>/api/bulkImportFullTexts?provenance=\<provenance\>&bulkImportDir=\<bulkImportDir\>&shouldDeleteFilesOnFinish={true|false}** <br>
This endpoint loads the right configuration with the help of the "provenance" parameter, delegates the processing to a background thread and immediately returns a message with useful information, including the "reportFileID", which can be used at any moment to request a report about the progress of the bulk-import procedure.<br> This endpoint loads the right configuration with the help of the "provenance" parameter, delegates the processing to a background thread and immediately returns a message with useful information, including the "reportFileID", which can be used at any moment to request a report about the progress of the bulk-import procedure. Use the **_HTTP POST_** method to access the endpoint.<br>
The processing job starts running after 30-60 minutes and processes the full-texts files inside the given directory, in the following way: it generates the OpenAIRE-IDs, uploads the files to the S3 Object Store, generates and stores the "payload" records in the database. If it is requested by the user, it removes the successfully imported full-texts from the directory. The processing job starts running after 30-60 minutes and processes the full-texts files inside the given directory, in the following way: it generates the OpenAIRE-IDs, uploads the files to the S3 Object Store, generates and stores the "payload" records in the database. If it is requested by the user, it removes the successfully imported full-texts from the directory.
- "**getBulkImportReport**" endpoint: **http://\<IP\>:\<PORT\>/api/getBulkImportReport?id=\<reportFileID\>** <br> - "**getBulkImportReport**" endpoint: **http://\<IP\>:\<PORT\>/api/getBulkImportReport?id=\<reportFileID\>** <br>
This endpoint returns the bulkImport report, which corresponds to the given reportFileID, in JSON format. This endpoint returns the bulkImport report, which corresponds to the given reportFileID, in JSON format.
@ -68,7 +68,9 @@ For managing and generating data, we use [**Impala**](https://impala.apache.org/
This endpoint specifies that the Workers will not shut down, and sends "cancelShutdownWorker" requests to all the Workers which are actively participating in the Service (have not shut down yet), so that they can continue to request assignments. This endpoint specifies that the Workers will not shut down, and sends "cancelShutdownWorker" requests to all the Workers which are actively participating in the Service (have not shut down yet), so that they can continue to request assignments.
<br> <br>
Note: The Shutdown Service API is accessible by the Controller's host machine. Notes:
- The Shutdown Service API is accessible by the Controller's host machine.<br>
- Use the **_HTTP POST_** method to access the endpoints.
<br> <br>
<br> <br>