D-Net
/
openaire-graph-docs
6
1
Fork 5
Code Issues 5 Pull Requests 2 Packages Projects Releases 67 Wiki Activity

10084: added cursor pagination. & date range in yyyy format #81

Merged
claudio.atzori merged 1 commits from 10084/cursor_pagination_date_format into main 2024-10-11 15:07:42 +02:00
Conversation 0 Commits 1 Files Changed 3 +131 -86
3 changed files with 131 additions and 86 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit dbb8809caf - Show all commits

162
docs/apis/graph-api/searching-entities/filtering-search-results.md
View File

@ -35,47 +35,48 @@ This section provides an overview of the available parameters for each entity ty
The following query parameters are available for research products:
| **Parameter** | **Description** |
|-------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **search** | Search in the content of the research product. |
| **mainTitle** | Search in the research product's main title. |
| **description** | Search in the research product's description. |
| **id** | The OpenAIRE id of the research product. |
| **pid** | The persistent identifier of the research product. |
| **originalId** | The identifier of the record at the original sources. |
| **type** | The type of the research product. One of `publication`, `dataset`, `software`, or `other` |
| **fromPublicationDate** | Gets the research products whose publication date is greater than or equal to the given date. A date formatted as `YYYY-MM-DD` |
| **toPublicationDate** | Gets the research products whose publication date is less than or equal to the given date. A date formatted as `YYYY-MM-DD` |
| **subjects** | List of subjects associated to the research product. |
| **countryCode** | The country code for the country associated with the research product. |
| **authorFullName** | The full name of the authors involved in producing this research product. |
| **authorOrcid** | The ORCiD of the authors involved in producing this research product. |
| **publisher** | The name of the entity that holds, archives, publishes prints, distributes, releases, issues, or produces the resource.
| **bestOpenAccessRightLabel** | The best open access rights among the research product's instances. One of `OPEN SOURCE`, `OPEN`, `EMBARGO`, `RESTRICTED`, `CLOSED`, `UNKNOWN` |
| **influenceClass** | Citation-based indicator that reflects the overall impact of a research product. Please, choose a class among `C1`, `C2`, `C3`, `C4`, or `C5` for top 0.01%, top 0.1%, top 1%, top 10%, and average in terms of influence respectively. |
| **impulseClass** | Citation-based indicator that reflects the initial momentum of a research product directly after its publication. Please, choose a class among `C1`, `C2`, `C3`, `C4`, or `C5` for top 0.01%, top 0.1%, top 1%, top 10%, and average in terms of impulse respectively
| **popularityClass** | Citation-based indicator that reflects current impact or attention of a research product. Please, choose a class among `C1`, `C2`, `C3`, `C4`, or `C5` for top 0.01%, top 0.1%, top 1%, top 10%, and average in terms of popularity respectively.
| **citationCountClass** | Citation-based indicator that reflects the overall impact of a research product by summing all its citations. Please, choose a class among `C1`, `C2`, `C3`, `C4`, or `C5` for top 0.01%, top 0.1%, top 1%, top 10%, and average in terms of citation count respectively.
| **instanceType** `[Only for publications]` | Retrieve publications of the given instance type. Check [here](http://api.openaire.eu/vocabularies/dnet:publication_resource) for all possible instance type values. |
| **sdg** `[Only for publications]` | Retrieves publications classified with the respective Sustainable Development Goal number. Integer in the range [1, 17] |
| **fos** `[Only for publications]` | Retrieves publications classified with a given Field of Science (FOS). A FOS classification identifier (see [here](https://explore.openaire.eu/assets/common-assets/vocabulary/fos.json) for details). |
| **isPeerReviewed** `[Only for publications]` | Indicates whether the publications are peerReviewed or not. (Boolean) |
| **isInDiamondJournal** `[Only for publications]` | Indicates whether the publication was published in a diamond journal or not. (Boolean) |
| **isPubliclyFunded** `[Only for publications]` | Indicates whether the publication was publicly funded or not. (Boolean) |
| **isGreen** `[Only for publications]` | Indicates whether the publication was published following the green open access model. (Boolean) |
| **openAccessColor** `[Only for publications]` | Specifies the Open Access color of the publication. One of `bronze`, `gold`, or `hybrid` |
| **relOrganizationId** | Retrieve research products connected to the organization (with OpenAIRE id). |
| **relCommunityId** | Retrieve research products connected to the community (with OpenAIRE id). |
| **relProjectId** | Retrieve research products connected to the project (with OpenAIRE id). |
| **relProjectCode** | Retrieve research products connected to the project with code. |
| **hasProjectRel** | Retrieve research products that are connected to a project. (Boolean) |
| **relProjectFundingShortName**| Retrieve research products connected to a project that has a funder with the given short name. |
| **relProjectFundingStreamId** | Retrieve research products connected to a project that has the given funding identifier. |
| **relHostingDataSourceId** | Retrieve research products hosted by the data source (with OpenAIRE id). |
| **relCollectedFromDatasourceId**| Retrieve research products collected from the data source (with OpenAIRE id). |
| **debugQuery** | Retrieve debug information for the search query. (Boolean) |
| **page** | Page number of the results. (Integer) |
| **pageSize** | Number of results per page. Integer in the range [1, 100] |
| **Parameter** | **Description** |
|-------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **search** | Search in the content of the research product. |
| **mainTitle** | Search in the research product's main title. |
| **description** | Search in the research product's description. |
| **id** | The OpenAIRE id of the research product. |
| **pid** | The persistent identifier of the research product. |
| **originalId** | The identifier of the record at the original sources. |
| **type** | The type of the research product. One of `publication`, `dataset`, `software`, or `other` |
| **fromPublicationDate** | Gets the research products whose publication date is greater than or equal to the given date. A date formatted as `ΥΥΥΥ` or `YYYY-MM-DD` |
| **toPublicationDate** | Gets the research products whose publication date is less than or equal to the given date. A date formatted as `YYYY` or `YYYY-MM-DD` |
| **subjects** | List of subjects associated to the research product. |
| **countryCode** | The country code for the country associated with the research product. |
| **authorFullName** | The full name of the authors involved in producing this research product. |
| **authorOrcid** | The ORCiD of the authors involved in producing this research product. |
| **publisher** | The name of the entity that holds, archives, publishes prints, distributes, releases, issues, or produces the resource. |
| **bestOpenAccessRightLabel** | The best open access rights among the research product's instances. One of `OPEN SOURCE`, `OPEN`, `EMBARGO`, `RESTRICTED`, `CLOSED`, `UNKNOWN` |
| **influenceClass** | Citation-based indicator that reflects the overall impact of a research product. Please, choose a class among `C1`, `C2`, `C3`, `C4`, or `C5` for top 0.01%, top 0.1%, top 1%, top 10%, and average in terms of influence respectively. |
| **impulseClass** | Citation-based indicator that reflects the initial momentum of a research product directly after its publication. Please, choose a class among `C1`, `C2`, `C3`, `C4`, or `C5` for top 0.01%, top 0.1%, top 1%, top 10%, and average in terms of impulse respectively |
| **popularityClass** | Citation-based indicator that reflects current impact or attention of a research product. Please, choose a class among `C1`, `C2`, `C3`, `C4`, or `C5` for top 0.01%, top 0.1%, top 1%, top 10%, and average in terms of popularity respectively. |
| **citationCountClass** | Citation-based indicator that reflects the overall impact of a research product by summing all its citations. Please, choose a class among `C1`, `C2`, `C3`, `C4`, or `C5` for top 0.01%, top 0.1%, top 1%, top 10%, and average in terms of citation count respectively. |
| **instanceType** `[Only for publications]` | Retrieve publications of the given instance type. Check [here](http://api.openaire.eu/vocabularies/dnet:publication_resource) for all possible instance type values. |
| **sdg** `[Only for publications]` | Retrieves publications classified with the respective Sustainable Development Goal number. Integer in the range [1, 17] |
| **fos** `[Only for publications]` | Retrieves publications classified with a given Field of Science (FOS). A FOS classification identifier (see [here](https://explore.openaire.eu/assets/common-assets/vocabulary/fos.json) for details). |
| **isPeerReviewed** `[Only for publications]` | Indicates whether the publications are peerReviewed or not. (Boolean) |
| **isInDiamondJournal** `[Only for publications]` | Indicates whether the publication was published in a diamond journal or not. (Boolean) |
| **isPubliclyFunded** `[Only for publications]` | Indicates whether the publication was publicly funded or not. (Boolean) |
| **isGreen** `[Only for publications]` | Indicates whether the publication was published following the green open access model. (Boolean) |
| **openAccessColor** `[Only for publications]` | Specifies the Open Access color of the publication. One of `bronze`, `gold`, or `hybrid` |
| **relOrganizationId** | Retrieve research products connected to the organization (with OpenAIRE id). |
| **relCommunityId** | Retrieve research products connected to the community (with OpenAIRE id). |
| **relProjectId** | Retrieve research products connected to the project (with OpenAIRE id). |
| **relProjectCode** | Retrieve research products connected to the project with code. |
| **hasProjectRel** | Retrieve research products that are connected to a project. (Boolean) |
| **relProjectFundingShortName**| Retrieve research products connected to a project that has a funder with the given short name. |
| **relProjectFundingStreamId** | Retrieve research products connected to a project that has the given funding identifier. |
| **relHostingDataSourceId** | Retrieve research products hosted by the data source (with OpenAIRE id). |
| **relCollectedFromDatasourceId**| Retrieve research products collected from the data source (with OpenAIRE id). |
| **debugQuery** | Retrieve debug information for the search query. (Boolean) |
| **page** | Page number of the results. (Integer) |
| **pageSize** | Number of results per page. Integer in the range [1, 100] |
| **cursor** | Cursor-based pagination. Initial value: `cursor=*` |
| **sortBy** | The field to set the sorting order of the results. Should be provided in the format `fieldname sortDirection`, where the `sortDirection` can be either `ASC` for ascending order or `DESC` for descending order and `fielaname` is one of `relevance`, `publicationDate`, `dateOfCollection`, `influence`, `popularity`, `citationCount`, `impulse`. Multiple sorting parameters should be comma-separated. |
@ -83,20 +84,21 @@ The following query parameters are available for research products:
The following query parameters are available for organizations:
| **Parameter** | **Description** |
|----------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|**search** | Search in the content of the organization. |
|**legalName** |The legal name of the organization. |
|**legalShortName** |The legal name of the organization in short form. |
|**id** |The OpenAIRE id of the organization. |
|**pid** |The persistent identifier of the organization. |
|**countryCode** |The country code of the organization. |
|**relCommunityId** |Retrieve organizations connected to the community (with OpenAIRE id). |
|**relCollectedFromDatasourceId**|Retrieve organizations collected from the data source (with OpenAIRE id). |
|**debugQuery** |Retrieve debug information for the search query. |
|**page** |Page number of the results. |
|**pageSize** |Number of results per page. |
|**sortBy** |The field to set the sorting order of the results. Should be provided in the format `fieldname sortDirection`, where the `sortDirection` can be either `ASC` for ascending order or `DESC` for descending order - organizations can only be sorted by `relevance`.|
| **Parameter** | **Description** |
|----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|**search** | Search in the content of the organization. |
|**legalName** | The legal name of the organization. |
|**legalShortName** | The legal name of the organization in short form. |
|**id** | The OpenAIRE id of the organization. |
|**pid** | The persistent identifier of the organization. |
|**countryCode** | The country code of the organization. |
|**relCommunityId** | Retrieve organizations connected to the community (with OpenAIRE id). |
|**relCollectedFromDatasourceId**| Retrieve organizations collected from the data source (with OpenAIRE id). |
|**debugQuery** | Retrieve debug information for the search query. |
|**page** | Page number of the results. |
|**pageSize** | Number of results per page. |
| **cursor** | Cursor-based pagination. Initial value: `cursor=*` |
|**sortBy** | The field to set the sorting order of the results. Should be provided in the format `fieldname sortDirection`, where the `sortDirection` can be either `ASC` for ascending order or `DESC` for descending order - organizations can only be sorted by `relevance`. |
### Data sources
@ -120,6 +122,7 @@ The following query parameters are available for data sources:
|**debugQuery** |Retrieve debug information for the search query. |
|**page** |Page number of the results. |
|**pageSize** |Number of results per page. |
| **cursor** | Cursor-based pagination. Initial value: `cursor=*` |
|**sortBy** |The field to set the sorting order of the results. Should be provided in the format `fieldname sortDirection`, where the `sortDirection` can be either `ASC` for ascending order or `DESC` for descending order - data sources can only be sorted by `relevance`.|
@ -127,30 +130,31 @@ The following query parameters are available for data sources:
The following query parameters are available for projects:
| **Parameter** | **Description** |
|----------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|**search** | Search in the content of the projects. |
|**title** |Search in the project's title. |
|**keywords** |The project's keywords. |
|**id** |The OpenAIRE id of the project. |
|**code** |The grant agreement (GA) code of the project. |
|**acronym** |Project's acronym. |
|**callIdentifier** |The identifier of the research call. |
|**fundingShortName** |The short name of the funder. |
|**fundingStreamId** |The identifier of the funding stream. |
|**fromStartDate** |Gets the projects with start date greater than or equal to the given date. Please provide a date formatted as `YYYY-MM-DD`. |
|**toStartDate** |Gets the projects with start date less than or equal to the given date. Please provide a date formatted as `YYYY-MM-DD`. |
|**fromEndDate** |Gets the projects with end date greater than or equal to the given date. Please provide a date formatted as `YYYY-MM-DD`. |
|**toEndDate** |Gets the projects with end date less than or equal to the given date. Please provide a date formatted as `YYYY-MM-DD`. |
|**relOrganizationName** |The name or short name of the related organization. |
|**relOrganizationId** |The organization identifier of the related organization. |
|**relCommunityId** |Retrieve projects connected to the community (with OpenAIRE id). |
|**relOrganizationCountryCode** |The country code of the related organizations. |
|**relCollectedFromDatasourceId**|Retrieve projects collected from the data source (with OpenAIRE id). |
|**debugQuery** |Retrieve debug information for the search query. |
|**page** |Page number of the results. |
|**pageSize** |Number of results per page. |
|**sortBy** |The field to set the sorting order of the results. Should be provided in the format `fieldname sortDirection`, where the `sortDirection` can be either `ASC` for ascending order or `DESC` for descending order and `fielaname` is one of `relevance`, `startDate`, `endDate`. Multiple sorting parameters should be comma-separated.|
| **Parameter** | **Description** |
|----------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|**search** | Search in the content of the projects. |
|**title** | Search in the project's title. |
|**keywords** | The project's keywords. |
|**id** | The OpenAIRE id of the project. |
|**code** | The grant agreement (GA) code of the project. |
|**acronym** | Project's acronym. |
|**callIdentifier** | The identifier of the research call. |
|**fundingShortName** | The short name of the funder. |
|**fundingStreamId** | The identifier of the funding stream. |
|**fromStartDate** | Gets the projects with start date greater than or equal to the given date. Please provide a date formatted as `YYYY` or `YYYY-MM-DD`. |
|**toStartDate** | Gets the projects with start date less than or equal to the given date. Please provide a date formatted as `YYYY` or `YYYY-MM-DD`. |
|**fromEndDate** | Gets the projects with end date greater than or equal to the given date. Please provide a date formatted as `YYYY` or `YYYY-MM-DD`. |
|**toEndDate** | Gets the projects with end date less than or equal to the given date. Please provide a date formatted as `YYYY` or `YYYY-MM-DD`. |
|**relOrganizationName** | The name or short name of the related organization. |
|**relOrganizationId** | The organization identifier of the related organization. |
|**relCommunityId** | Retrieve projects connected to the community (with OpenAIRE id). |
|**relOrganizationCountryCode** | The country code of the related organizations. |
|**relCollectedFromDatasourceId**| Retrieve projects collected from the data source (with OpenAIRE id). |
|**debugQuery** | Retrieve debug information for the search query. |
|**page** | Page number of the results. |
|**pageSize** | Number of results per page. |
| **cursor** | Cursor-based pagination. Initial value: `cursor=*` |
|**sortBy** | The field to set the sorting order of the results. Should be provided in the format `fieldname sortDirection`, where the `sortDirection` can be either `ASC` for ascending order or `DESC` for descending order and `fielaname` is one of `relevance`, `startDate`, `endDate`. Multiple sorting parameters should be comma-separated. |
## Using logical operators

4
docs/apis/graph-api/searching-entities/searching-entities.md
View File

@ -31,14 +31,14 @@ The response of the aforementioned endpoints is an object of the following type:
...
]
}
```
It contains a `header` object with the following fields:
- `numFound`: the total number of entities found
- `maxScore`: the maximum relevance score of the search results
- `queryTime`: the time in milliseconds that the search took
- `page`: the current page of the search results
- `page`: the current page of the search results (when using basic pagination)
- `pageSize`: the number of entities per page
- `nextCursor`: the next page cursor (when using cursor-based pagination, see: [paging](./sorting-and-paging.md#paging)
Finally, the `results` field contains an array of entities of the corresponding type (i.e., [Research product](../../../data-model/entities/research-product.md), [Organization](../../../data-model/entities/organization.md), [Data Source](../../../data-model/entities/data-source.md), or [Project](../../../data-model/entities/project.md)).

51
docs/apis/graph-api/searching-entities/sorting-and-paging.md
View File

@ -31,16 +31,57 @@ Example:
[https://api-beta.openaire.eu/graph/researchProducts?search=COVID-19&sortBy=publicationDate ASC,popularity DESC](https://api-beta.openaire.eu/graph/researchProducts?search=COVID-19&sortBy=publicationDate%20ASC,popularity%20DESC)
## Paging
## Paging
The OpenAIRE Graph API supports paging through the use of `page` and `pageSize` parameters, enabling you to specify which part of the result set to retrieve and how many results per page.
The OpenAIRE Graph API supports basic and cursor-based pagination. In basic pagination, `page` and `pageSize` parameters are used, enabling you to specify which part of the result set to retrieve and how many results per page.
### Offset-based paging
Offset-based paging should be used to retrieve a small dataset only (up to 10000 records).
* `page`: Specifies the page number of the results you want to retrieve. Page numbering starts from 1.
* `pageSize`: Defines the number of results to be returned per page. This helps limit the amount of data returned in a single request, making it easier to process.
Example:
Example:
- Get the top 10 most influential research products that contain the phrase "knowledge graphs":
[https://api-beta.openaire.eu/graph/researchProducts?search="knowledge graphs"&page=1&pageSize=10&sortBy=influence DESC](https://api-beta.openaire.eu/graph/researchProducts?search=%22knowledge%20graphs%22&page=1&pageSize=10&sortBy=influence%20DESC)
[https://api-beta.openaire.eu/graph/researchProducts?search="knowledge graphs"&page=1&pageSize=10&sortBy=influence DESC](https://api-beta.openaire.eu/graph/researchProducts?search=%22knowledge%20graphs%22&page=1&pageSize=10&sortBy=influence%20DESC)
response:
```json
{
header: {
numFound: 36818386,
maxScore: 1,
queryTime: 21,
page: 1,
pageSize: 10
},
results: [
...
]
}
```
### Cursor-based paging
Cursor should be used when it is required to retrieve a big dataset (more than 10000 records).
* `cursor`: Cursor-based pagination. Initial value: `cursor=*`.
Example:
- [https://api-beta.openaire.eu/graph/researchProducts?search="knowledge graphs"&pageSize=10&cursor=*&sortBy=influence DESC](https://api-beta.openaire.eu/graph/researchProducts?search=%22knowledge%20graphs%22&pageSize=10&cursor=*&sortBy=influence%20DESC)
response:
```json
{
header: {
numFound: 36818386,
maxScore: 1,
queryTime: 21,
pageSize: 10,
nextCursor: "AoI/D2M2NGU1YjVkNTQ4Nzo6NjlmZTBmNjljYzM4YTY1MjI5YjM3ZDRmZmIyMTU1NDAIP4AAAA=="
},
results: [
...
]
}
```
Use `nextCursor` value, to get the next page of results.