D-Net
/
openaire-graph-docs
6
1
Fork 5
Code Issues 5 Pull Requests 2 Packages Projects Releases 67 Wiki Activity

Eliminate the "ambiguous unicode character" warning on Gitea.

This commit is contained in:
Lampros Smyrnaios 2023-07-03 18:07:01 +03:00
parent a4f15a3f83
commit 1c668b7fd8
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/graph-production-workflow/enrichment-by-mining/enrichment-by-mining.md
View File

@ -5,7 +5,7 @@ import DocCardList from '@theme/DocCardList';
**OpenAIRE** collects the full-texts of the publications, in order to apply TDM (Text and Data Mining) algorithms on them and enrich the Graph with inference links. **OpenAIRE** collects the full-texts of the publications, in order to apply TDM (Text and Data Mining) algorithms on them and enrich the Graph with inference links.
The collection of the full-texts is handled by the internal **PDF Aggregation Service**. This service uses the publications’ urls, from the OpenAIRE Graph and state-of-the-art algorithms, to crawl the web and try to locate and download the full-texts of the open access publications, while focusing on the most recent ones. It respects the servers of the repositories and publishers and avoids overloading them. The collection of the full-texts is handled by the internal **PDF Aggregation Service**. This service uses the publications' urls, from the OpenAIRE Graph and state-of-the-art algorithms, to crawl the web and try to locate and download the full-texts of the open access publications, while focusing on the most recent ones. It respects the servers of the repositories and publishers and avoids overloading them.
The service is orchestrating a distributed execution system, on the cloud, with multiple microservices running in parallel, in order to efficiently process and download a large number of publications. The microservices store the generated report records for the publications, in a database, and the full-texts in an S3 Object Store. The service is orchestrating a distributed execution system, on the cloud, with multiple microservices running in parallel, in order to efficiently process and download a large number of publications. The microservices store the generated report records for the publications, in a database, and the full-texts in an S3 Object Store.