From 1c9ff0669ad0ff316b4427414502e80473d3dcb5 Mon Sep 17 00:00:00 2001 From: LSmyrnaios Date: Tue, 14 Nov 2023 18:45:45 +0200 Subject: [PATCH 1/7] - Add information in the README, about the "Building", "Deployment", "Installation" and "Configuration" of the project. - Eliminate a warning in the POM file, about a missing tag. --- README.md | 234 ++++++++++++++++++++++++++++++++++++++++++++++++++++-- pom.xml | 1 + 2 files changed, 230 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 169b22b..5cb53f4 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,238 @@ # uoa-repository-manager-service (Provide backend) +## Introduction +## Architecture + + +
+
+
+ +## Building +The backend is a [Maven](https://maven.apache.org/index.html) project. It has been tested using Java 8. +
+ +#### Manual Build Instructions + +###### Requirements: +- Java 8 +- Apache Maven 3+ + +###### Build Instructions: +1. Clone the repository and move inside the directory +
`git clone https://code-repo.d4science.org/MaDgIK/uoa-repository-manager-service.git && cd uoa-repository-manager-service` +2. Provide all not-set or redacted configurations, inside the **src/main/resources/application.yml** file. +3. Build Maven project +
`mvn clean package` +
Produces the file "./target/**uoa-repository-manager-service.jar**" which can be run with: - Run the app with: `java -jar ./target/uoa-repository-manager-service.jar` +

+

+ +#### Build using Docker +The repository contains a **Dockerfile** which can be used to build an image containing the compiled project. + +###### Requirements: +- Docker + +###### Build Instructions: +1. Clone the repository and move inside the directory +
`git clone https://code-repo.d4science.org/MaDgIK/uoa-repository-manager-service.git && cd uoa-repository-manager-service` +2. Provide all not-set or redacted configurations, inside the **src/main/resources/application.yml** file. +3. Build Docker image +
`docker build . -t ` + + +## Deployment + +### Prerequisites +* [PostgreSQL 9.5+](https://www.postgresql.org/) + [...] + + +### Instructions +Execute `java -jar ./target/uoa-repository-manager-service.jar` + + +## Installation + +### Prerequisites +* [PostgreSQL 9.5+](https://www.postgresql.org/) + + +### Infrastructure Installation and Configuration +Install all software found in the list of [Prerequisites](#Prerequisites) using the official documentation. + + +#### PostgreSQL - Configuration [...] -## Install and run: -- Run **git clone** and then **cd uoa-repository-manager-service**. -- Provide all not-set or redacted configurations, inside the **src/main/resources/application.yml** file. -- Build the app with: `mvn clean install` -- Run the app with: `java -jar ./target/uoa-repository-manager-service.jar` +### Installation +No additional installation steps are needed. +## Configuration + +The configuration can be set inside the **src/main/resources/application.yml** file. + +### Server-related configuration. +``` +server: + port: 8480 + servlet: + context-path: /uoa-repository-manager-service +``` + +### Swagger UI - related configuration +``` +springdoc: + swagger-ui: + disable-swagger-default-url: true + path: /swagger-ui.html + displayRequestDuration: true + api-docs: + path: /api-docs +``` + +### Datasource -related configuration +``` +spring: + jpa: + hibernate: + ddl-auto: update + datasource: + url: ${services.provide.db.url} + username: ${services.provide.db.username} + password: ${services.provide.db.password} + driverClassName: ${services.provide.db.driverClassName} +``` + +### Configurations about the provide service. +``` +services: + openaireServicesBaseUrl: https://beta.services.openaire.eu + provide: + dev-machine: 88.197.53.71 # VM-71 +``` + +#### Configuration about Authentication and authorization infrastructure (AAI). +``` + aai: + baseURL: https://aai.openaire.eu + oidc: + domain: .openaire.eu # use empty value for local, otherwise: ".openaire.eu" + id: XX + issuer: ${services.provide.aai.baseURL}/oidc/ + redirectURL: http://localhost:${server.port}${server.servlet.context-path}/openid_connect_login + secret: XX + webURL: http://localhost:4200/join + registry: + coid: XX + username: XX + password: XX + production: false + url: ${services.provide.aai.baseURL}/registry/ +``` + +#### [...] +``` + adminEmail: XX + analyticsURL: https://analytics.openaire.eu/addsite.php? + baseUrl: ${services.openaireServicesBaseUrl}/openaire +``` + +#### Broker's configuration +``` + broker: + api: api/ + openaire: openaireBroker + port: 443 + url: https://beta.broker.openaire.eu +``` + +#### Client's configuration +``` + clients: + dsm: ${services.provide.baseUrl} + search: ${services.openaireServicesBaseUrl}/search/v2/api + usageEvents: http://beta.lbs.openaire.eu:8080/ajax/summary + usagestats: ${services.openaireServicesBaseUrl}/usagestats +``` + +#### Database configuration for Provide. +``` + db: + driverClassName: org.postgresql.Driver + password: dnetPwd + url: jdbc:postgresql://${services.provide.dev-machine}:5432/repomanager + username: dnet +``` + +#### Configuration for the IS LookUp service. +``` + iSLookUpService: + url: https://dev-openaire.d4science.org:443/is/services/isLookUp +``` + +#### Mail-server configuration. +``` + mail: + authenticate: true + debug: false + from: XX + host: smtp.gmail.com + mode: ssl + password: XX + port: 465 + replyTo: XX + username: XX +``` + +#### Redis's configuration +``` + redis: + host: vereniki.athenarc.gr + password: XX + port: 6379 +``` + +#### [...] +``` + topic_types: + url: ${services.openaireServicesBaseUrl}/provision/mvc/vocabularies/dnet:topic_types.json # TODO - Check this! The requested json file does not exist in the DEV-url below) + usageStatisticsDiagramsBaseURL: https://beta.openaire.eu/stats3/ + usageStatisticsNumbersBaseURL: ${services.openaireServicesBaseUrl}/usagestats/datasources/ + usagestats: + adminEmail: XX + sushiliteEndpoint: ${services.openaireServicesBaseUrl}/usagestats/sushilite/ + sushiliteR5Endpoint: ${services.openaireServicesBaseUrl}/usagestats_r5/sushilite/r5/ +``` + +#### Validator - related configuration. +``` + validator: + results: + url: https://beta.provide.openaire.eu/compatibility/browseHistory/ + validatorService: + url: http://${services.provide.dev-machine}:8080/uoa-validator-service/services/validatorWebService +``` + +
+
+
+ + +## Security +## Maintenance +## Recovery +## References + + +
+
+
+ +--- + ### Notes for Swagger-UI: - Access it through this url: http://localhost:8480/uoa-repository-manager-service/swagger-ui/index.html - In order to request data from most endpoints, you have to be a "REGISTERED_USER", otherwise you will get a 403 error code. diff --git a/pom.xml b/pom.xml index cfa3754..8dc28ea 100644 --- a/pom.xml +++ b/pom.xml @@ -7,6 +7,7 @@ org.springframework.boot spring-boot-starter-parent 2.7.17 + 4.0.0 From 7853494a5b855cc71725eaa2b7bd32fe4ea2db16 Mon Sep 17 00:00:00 2001 From: LSmyrnaios Date: Wed, 15 Nov 2023 13:01:56 +0200 Subject: [PATCH 2/7] Remove obsolete paragraph. --- README.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/README.md b/README.md index 60f8d0b..f8fb33e 100644 --- a/README.md +++ b/README.md @@ -66,9 +66,6 @@ Install all software found in the list of [Prerequisites](#Prerequisites) using #### PostgreSQL - Configuration [...] -### Installation -No additional installation steps are needed. - ## Configuration From 0deeb00ee5a42659d17a1768a722fca3e7d4998f Mon Sep 17 00:00:00 2001 From: LSmyrnaios Date: Wed, 13 Mar 2024 18:20:45 +0200 Subject: [PATCH 3/7] - Replaced DSM's "AggregationInfo" with "CollectionInfoV2" in "eu.dnetlib.repo.manager.domain.AggregationInfo". - Cleanup related dependencies. Co-authored-by: Konstantinos Spyrou --- pom.xml | 19 +++++++------------ .../controllers/DashboardController.java | 6 +----- .../controllers/RepositoryController.java | 2 -- .../repo/manager/domain/AggregationInfo.java | 5 +++-- .../domain/CollectionMonitorSummary.java | 2 -- .../manager/service/AggregationService.java | 2 +- .../service/AggregationServiceImpl.java | 2 +- .../manager/service/DashboardServiceImpl.java | 2 +- 8 files changed, 14 insertions(+), 26 deletions(-) diff --git a/pom.xml b/pom.xml index 8dc28ea..1c89ead 100644 --- a/pom.xml +++ b/pom.xml @@ -76,21 +76,16 @@ - org.apache.solr - solr-solrj - 9.4.0 + eu.dnetlib.dhp + dnet-exporter-model + 3.5.0 + - eu.dnetlib.dhp - dnet-exporter-api - [3.3.3-SNAPSHOT, ) - - - eu.dnetlib - dnet-index-solr-common - - + io.micrometer + micrometer-registry-prometheus + 1.12.4 diff --git a/src/main/java/eu/dnetlib/repo/manager/controllers/DashboardController.java b/src/main/java/eu/dnetlib/repo/manager/controllers/DashboardController.java index 5a1d7fb..115f449 100644 --- a/src/main/java/eu/dnetlib/repo/manager/controllers/DashboardController.java +++ b/src/main/java/eu/dnetlib/repo/manager/controllers/DashboardController.java @@ -1,10 +1,6 @@ package eu.dnetlib.repo.manager.controllers; -import eu.dnetlib.enabling.datasources.common.AggregationInfo; -import eu.dnetlib.repo.manager.domain.BrokerSummary; -import eu.dnetlib.repo.manager.domain.CollectionMonitorSummary; -import eu.dnetlib.repo.manager.domain.RepositorySummaryInfo; -import eu.dnetlib.repo.manager.domain.UsageSummary; +import eu.dnetlib.repo.manager.domain.*; import eu.dnetlib.repo.manager.exception.BrokerException; import eu.dnetlib.repo.manager.exception.RepositoryServiceException; import eu.dnetlib.repo.manager.service.*; diff --git a/src/main/java/eu/dnetlib/repo/manager/controllers/RepositoryController.java b/src/main/java/eu/dnetlib/repo/manager/controllers/RepositoryController.java index d36a915..441f16e 100644 --- a/src/main/java/eu/dnetlib/repo/manager/controllers/RepositoryController.java +++ b/src/main/java/eu/dnetlib/repo/manager/controllers/RepositoryController.java @@ -1,7 +1,6 @@ package eu.dnetlib.repo.manager.controllers; import eu.dnetlib.domain.data.PiwikInfo; -import eu.dnetlib.enabling.datasources.common.AggregationInfo; import eu.dnetlib.repo.manager.domain.*; import eu.dnetlib.repo.manager.domain.dto.RepositoryTerms; import eu.dnetlib.repo.manager.domain.dto.User; @@ -9,7 +8,6 @@ import eu.dnetlib.repo.manager.exception.RepositoryServiceException; import eu.dnetlib.repo.manager.exception.ResourceNotFoundException; import eu.dnetlib.repo.manager.service.AggregationService; import eu.dnetlib.repo.manager.service.PiWikService; -import eu.dnetlib.repo.manager.service.PiWikServiceImpl; import eu.dnetlib.repo.manager.service.RepositoryService; import eu.dnetlib.repo.manager.service.security.AuthorizationService; import eu.dnetlib.repo.manager.utils.JsonUtils; diff --git a/src/main/java/eu/dnetlib/repo/manager/domain/AggregationInfo.java b/src/main/java/eu/dnetlib/repo/manager/domain/AggregationInfo.java index 2fa6a0e..bcadee5 100644 --- a/src/main/java/eu/dnetlib/repo/manager/domain/AggregationInfo.java +++ b/src/main/java/eu/dnetlib/repo/manager/domain/AggregationInfo.java @@ -1,7 +1,8 @@ package eu.dnetlib.repo.manager.domain; -// TODO: this class must be replaced by common dnet class (CollectionInfo / TransformationInfo / AggregationInfo) -public class AggregationInfo extends eu.dnetlib.enabling.datasources.common.AggregationInfo { +import eu.dnetlib.openaire.exporter.model.dsm.CollectionInfoV2; + +public class AggregationInfo extends CollectionInfoV2 { public AggregationInfo() { } diff --git a/src/main/java/eu/dnetlib/repo/manager/domain/CollectionMonitorSummary.java b/src/main/java/eu/dnetlib/repo/manager/domain/CollectionMonitorSummary.java index db279ea..c272118 100644 --- a/src/main/java/eu/dnetlib/repo/manager/domain/CollectionMonitorSummary.java +++ b/src/main/java/eu/dnetlib/repo/manager/domain/CollectionMonitorSummary.java @@ -1,7 +1,5 @@ package eu.dnetlib.repo.manager.domain; -import eu.dnetlib.enabling.datasources.common.AggregationInfo; - import java.util.List; public class CollectionMonitorSummary { diff --git a/src/main/java/eu/dnetlib/repo/manager/service/AggregationService.java b/src/main/java/eu/dnetlib/repo/manager/service/AggregationService.java index 8741201..eff9b26 100644 --- a/src/main/java/eu/dnetlib/repo/manager/service/AggregationService.java +++ b/src/main/java/eu/dnetlib/repo/manager/service/AggregationService.java @@ -1,6 +1,6 @@ package eu.dnetlib.repo.manager.service; -import eu.dnetlib.enabling.datasources.common.AggregationInfo; +import eu.dnetlib.repo.manager.domain.AggregationInfo; import org.json.JSONException; import java.util.List; diff --git a/src/main/java/eu/dnetlib/repo/manager/service/AggregationServiceImpl.java b/src/main/java/eu/dnetlib/repo/manager/service/AggregationServiceImpl.java index b6080d1..611e28f 100644 --- a/src/main/java/eu/dnetlib/repo/manager/service/AggregationServiceImpl.java +++ b/src/main/java/eu/dnetlib/repo/manager/service/AggregationServiceImpl.java @@ -1,7 +1,7 @@ package eu.dnetlib.repo.manager.service; -import eu.dnetlib.enabling.datasources.common.AggregationInfo; import eu.dnetlib.repo.manager.domain.AggregationHistoryResponse; +import eu.dnetlib.repo.manager.domain.AggregationInfo; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Value; diff --git a/src/main/java/eu/dnetlib/repo/manager/service/DashboardServiceImpl.java b/src/main/java/eu/dnetlib/repo/manager/service/DashboardServiceImpl.java index c84bb03..49098c0 100644 --- a/src/main/java/eu/dnetlib/repo/manager/service/DashboardServiceImpl.java +++ b/src/main/java/eu/dnetlib/repo/manager/service/DashboardServiceImpl.java @@ -1,6 +1,6 @@ package eu.dnetlib.repo.manager.service; -import eu.dnetlib.enabling.datasources.common.AggregationInfo; +import eu.dnetlib.repo.manager.domain.AggregationInfo; import eu.dnetlib.repo.manager.domain.MetricsNumbers; import eu.dnetlib.repo.manager.domain.RepositorySnippet; import eu.dnetlib.repo.manager.domain.RepositorySummaryInfo; From 6dfe11484e8415331ff3f37b1b21a907e98e5599 Mon Sep 17 00:00:00 2001 From: LSmyrnaios Date: Fri, 15 Mar 2024 13:58:38 +0200 Subject: [PATCH 4/7] - Add the Maven Central repository. - Add/update/fix/cleanup dependencies. - Add the "show-actuator"-config for springdoc. - Remove obsolete configuration. --- pom.xml | 71 +++++++++++-------- .../service/RepositoryServiceImpl.java | 2 + src/main/resources/application-context.xml | 42 ----------- src/main/resources/application.yml | 1 + 4 files changed, 45 insertions(+), 71 deletions(-) delete mode 100644 src/main/resources/application-context.xml diff --git a/pom.xml b/pom.xml index 1c89ead..42f43c1 100644 --- a/pom.xml +++ b/pom.xml @@ -6,7 +6,7 @@ org.springframework.boot spring-boot-starter-parent - 2.7.17 + 2.7.18 @@ -17,7 +17,7 @@ jar - 2.7.17 + 2.7.18 @@ -68,6 +68,24 @@ ${spring.boot.version} test + + org.springframework.boot + spring-boot-configuration-processor + ${spring.boot.version} + true + + + org.springframework.boot + spring-boot-devtools + ${spring.boot.version} + true + + + + org.springdoc + springdoc-openapi-ui + 1.8.0 + com.h2database @@ -78,7 +96,13 @@ eu.dnetlib.dhp dnet-exporter-model - 3.5.0 + [3.5.0, ) + + + org.springdoc + springdoc-openapi-common + + @@ -129,13 +153,6 @@ - - eu.dnetlib - dnet-openaire-usage-stats-sushilite-r5 - 1.0.0 - system - ${project.basedir}/libs/openaire-usage-stats-sushilite-r5-1.0.0.jar - eu.dnetlib dnet-openaire-usage-stats-sushilite @@ -157,6 +174,7 @@ + cglib cglib-nodep @@ -166,7 +184,7 @@ commons-io commons-io - 2.15.0 + 2.15.1 @@ -200,7 +218,7 @@ org.json json - 20231013 + 20240303 @@ -209,27 +227,11 @@ 1.6.2 - - org.springdoc - springdoc-openapi-webmvc-core - 1.7.0 - - - org.springdoc - springdoc-openapi-ui - 1.7.0 - - - org.springdoc - springdoc-openapi-security - 1.7.0 - - org.postgresql postgresql - 42.6.0 + 42.7.2 @@ -406,6 +408,15 @@ + + central + Maven Central + https://repo.maven.apache.org/maven2 + default + + false + + dnet45-bootstrap-snapshot D-Net 45 Bootstrap Snapshot @@ -460,6 +471,8 @@ false + + - - - - - - - - - - - \ No newline at end of file diff --git a/src/main/resources/application.yml b/src/main/resources/application.yml index 5b3a5d5..b017111 100644 --- a/src/main/resources/application.yml +++ b/src/main/resources/application.yml @@ -10,6 +10,7 @@ springdoc: displayRequestDuration: true api-docs: path: /api-docs + show-actuator: true spring: jpa: From c836de5761d2081866db23cc5c7167842da151a9 Mon Sep 17 00:00:00 2001 From: LSmyrnaios Date: Fri, 22 Mar 2024 11:51:22 +0200 Subject: [PATCH 5/7] Use the new DSM-2.0 API to obtain the "aggregation history" of a repository. --- pom.xml | 2 +- .../controllers/DashboardController.java | 6 +++- .../controllers/RepositoryController.java | 1 + .../domain/AggregationHistoryResponse.java | 29 ------------------- .../repo/manager/domain/AggregationInfo.java | 9 ------ .../domain/CollectionMonitorSummary.java | 2 ++ .../manager/service/AggregationService.java | 2 +- .../service/AggregationServiceImpl.java | 8 ++--- .../manager/service/DashboardServiceImpl.java | 4 +-- 9 files changed, 16 insertions(+), 47 deletions(-) delete mode 100644 src/main/java/eu/dnetlib/repo/manager/domain/AggregationHistoryResponse.java delete mode 100644 src/main/java/eu/dnetlib/repo/manager/domain/AggregationInfo.java diff --git a/pom.xml b/pom.xml index 42f43c1..afce7f3 100644 --- a/pom.xml +++ b/pom.xml @@ -96,7 +96,7 @@ eu.dnetlib.dhp dnet-exporter-model - [3.5.0, ) + [3.5.1, ) org.springdoc diff --git a/src/main/java/eu/dnetlib/repo/manager/controllers/DashboardController.java b/src/main/java/eu/dnetlib/repo/manager/controllers/DashboardController.java index 115f449..b0a9960 100644 --- a/src/main/java/eu/dnetlib/repo/manager/controllers/DashboardController.java +++ b/src/main/java/eu/dnetlib/repo/manager/controllers/DashboardController.java @@ -1,6 +1,10 @@ package eu.dnetlib.repo.manager.controllers; -import eu.dnetlib.repo.manager.domain.*; +import eu.dnetlib.openaire.exporter.model.dsm.AggregationInfo; +import eu.dnetlib.repo.manager.domain.BrokerSummary; +import eu.dnetlib.repo.manager.domain.CollectionMonitorSummary; +import eu.dnetlib.repo.manager.domain.RepositorySummaryInfo; +import eu.dnetlib.repo.manager.domain.UsageSummary; import eu.dnetlib.repo.manager.exception.BrokerException; import eu.dnetlib.repo.manager.exception.RepositoryServiceException; import eu.dnetlib.repo.manager.service.*; diff --git a/src/main/java/eu/dnetlib/repo/manager/controllers/RepositoryController.java b/src/main/java/eu/dnetlib/repo/manager/controllers/RepositoryController.java index 441f16e..b5c3013 100644 --- a/src/main/java/eu/dnetlib/repo/manager/controllers/RepositoryController.java +++ b/src/main/java/eu/dnetlib/repo/manager/controllers/RepositoryController.java @@ -1,6 +1,7 @@ package eu.dnetlib.repo.manager.controllers; import eu.dnetlib.domain.data.PiwikInfo; +import eu.dnetlib.openaire.exporter.model.dsm.AggregationInfo; import eu.dnetlib.repo.manager.domain.*; import eu.dnetlib.repo.manager.domain.dto.RepositoryTerms; import eu.dnetlib.repo.manager.domain.dto.User; diff --git a/src/main/java/eu/dnetlib/repo/manager/domain/AggregationHistoryResponse.java b/src/main/java/eu/dnetlib/repo/manager/domain/AggregationHistoryResponse.java deleted file mode 100644 index 65e932d..0000000 --- a/src/main/java/eu/dnetlib/repo/manager/domain/AggregationHistoryResponse.java +++ /dev/null @@ -1,29 +0,0 @@ -package eu.dnetlib.repo.manager.domain; - -import com.fasterxml.jackson.annotation.JsonAutoDetect; - -import java.util.List; - -@JsonAutoDetect -public class AggregationHistoryResponse extends Response { - - private List aggregationInfo; - - public AggregationHistoryResponse() { - // no-arg constructor - } - - public AggregationHistoryResponse(final List aggregationInfo) { - super(); - this.aggregationInfo = aggregationInfo; - } - - public List getAggregationInfo() { - return aggregationInfo; - } - - public void setAggregationInfo(final List aggregationInfo) { - this.aggregationInfo = aggregationInfo; - } -} - diff --git a/src/main/java/eu/dnetlib/repo/manager/domain/AggregationInfo.java b/src/main/java/eu/dnetlib/repo/manager/domain/AggregationInfo.java deleted file mode 100644 index bcadee5..0000000 --- a/src/main/java/eu/dnetlib/repo/manager/domain/AggregationInfo.java +++ /dev/null @@ -1,9 +0,0 @@ -package eu.dnetlib.repo.manager.domain; - -import eu.dnetlib.openaire.exporter.model.dsm.CollectionInfoV2; - -public class AggregationInfo extends CollectionInfoV2 { - - public AggregationInfo() { - } -} diff --git a/src/main/java/eu/dnetlib/repo/manager/domain/CollectionMonitorSummary.java b/src/main/java/eu/dnetlib/repo/manager/domain/CollectionMonitorSummary.java index c272118..704e1d3 100644 --- a/src/main/java/eu/dnetlib/repo/manager/domain/CollectionMonitorSummary.java +++ b/src/main/java/eu/dnetlib/repo/manager/domain/CollectionMonitorSummary.java @@ -1,5 +1,7 @@ package eu.dnetlib.repo.manager.domain; +import eu.dnetlib.openaire.exporter.model.dsm.AggregationInfo; + import java.util.List; public class CollectionMonitorSummary { diff --git a/src/main/java/eu/dnetlib/repo/manager/service/AggregationService.java b/src/main/java/eu/dnetlib/repo/manager/service/AggregationService.java index eff9b26..6d62a41 100644 --- a/src/main/java/eu/dnetlib/repo/manager/service/AggregationService.java +++ b/src/main/java/eu/dnetlib/repo/manager/service/AggregationService.java @@ -1,6 +1,6 @@ package eu.dnetlib.repo.manager.service; -import eu.dnetlib.repo.manager.domain.AggregationInfo; +import eu.dnetlib.openaire.exporter.model.dsm.AggregationInfo; import org.json.JSONException; import java.util.List; diff --git a/src/main/java/eu/dnetlib/repo/manager/service/AggregationServiceImpl.java b/src/main/java/eu/dnetlib/repo/manager/service/AggregationServiceImpl.java index 611e28f..7fc5e3a 100644 --- a/src/main/java/eu/dnetlib/repo/manager/service/AggregationServiceImpl.java +++ b/src/main/java/eu/dnetlib/repo/manager/service/AggregationServiceImpl.java @@ -1,7 +1,7 @@ package eu.dnetlib.repo.manager.service; -import eu.dnetlib.repo.manager.domain.AggregationHistoryResponse; -import eu.dnetlib.repo.manager.domain.AggregationInfo; +import eu.dnetlib.openaire.exporter.model.dsm.AggregationHistoryResponseV2; +import eu.dnetlib.openaire.exporter.model.dsm.AggregationInfo; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Value; @@ -41,7 +41,7 @@ public class AggregationServiceImpl implements AggregationService { logger.debug("Retrieving aggregations for repository with id : " + id); UriComponents uriComponents = getAggregationHistory(id); - AggregationHistoryResponse rs = restTemplate.getForObject(uriComponents.toUri(), AggregationHistoryResponse.class); + AggregationHistoryResponseV2 rs = restTemplate.getForObject(uriComponents.toUri(), AggregationHistoryResponseV2.class); return rs != null ? (List) rs.getAggregationInfo() : null; } @@ -73,7 +73,7 @@ public class AggregationServiceImpl implements AggregationService { private UriComponents getAggregationHistory(String repoId) { return UriComponentsBuilder - .fromHttpUrl(baseAddress + "/ds/aggregationhistory/") + .fromHttpUrl(baseAddress + "/dsm/2.0/aggregationhistory/") .path(repoId) .build().expand(repoId).encode(); } diff --git a/src/main/java/eu/dnetlib/repo/manager/service/DashboardServiceImpl.java b/src/main/java/eu/dnetlib/repo/manager/service/DashboardServiceImpl.java index 49098c0..7fa8718 100644 --- a/src/main/java/eu/dnetlib/repo/manager/service/DashboardServiceImpl.java +++ b/src/main/java/eu/dnetlib/repo/manager/service/DashboardServiceImpl.java @@ -1,6 +1,6 @@ package eu.dnetlib.repo.manager.service; -import eu.dnetlib.repo.manager.domain.AggregationInfo; +import eu.dnetlib.openaire.exporter.model.dsm.AggregationInfo; import eu.dnetlib.repo.manager.domain.MetricsNumbers; import eu.dnetlib.repo.manager.domain.RepositorySnippet; import eu.dnetlib.repo.manager.domain.RepositorySummaryInfo; @@ -49,7 +49,7 @@ public class DashboardServiceImpl implements DashboardService { repositorySummaryInfo.setRepositoryName(repoOfficialName); repositorySummaryInfo.setLogoURL(repository.getLogoUrl()); - //TODO getRepositoryAggregations returns only the 20 more recent items. Is it positive that we will find an indexed version there? + //TODO getRepositoryAggregations returns only the 20 more recent items. Is it possible that we will find an indexed version there? boolean isIndexedVersionFound = false; long start = System.currentTimeMillis(); List aggregationInfoList = aggregationService.getRepositoryAggregations(repoId, 0, 20); From cb458553f06204143a9beda949d532a5d8a4ba23 Mon Sep 17 00:00:00 2001 From: LSmyrnaios Date: Wed, 3 Apr 2024 14:59:43 +0300 Subject: [PATCH 6/7] - Update the "RepositoryServiceImpl.getCompatibilityClasses()" method to recognise additional "CRIS" modes. - Catch exceptions thrown in "RepositoryServiceImpl.searchApi()". - Code optimization. - Update dependencies. --- pom.xml | 6 +++--- .../manager/service/RepositoryServiceImpl.java | 16 +++++++++++----- 2 files changed, 14 insertions(+), 8 deletions(-) diff --git a/pom.xml b/pom.xml index afce7f3..4554a4f 100644 --- a/pom.xml +++ b/pom.xml @@ -184,7 +184,7 @@ commons-io commons-io - 2.15.1 + 2.16.0 @@ -204,7 +204,7 @@ org.glassfish.jersey.core jersey-client - 2.41 + 2.42 @@ -231,7 +231,7 @@ org.postgresql postgresql - 42.7.2 + 42.7.3 diff --git a/src/main/java/eu/dnetlib/repo/manager/service/RepositoryServiceImpl.java b/src/main/java/eu/dnetlib/repo/manager/service/RepositoryServiceImpl.java index 46b3d72..c8e3366 100644 --- a/src/main/java/eu/dnetlib/repo/manager/service/RepositoryServiceImpl.java +++ b/src/main/java/eu/dnetlib/repo/manager/service/RepositoryServiceImpl.java @@ -686,19 +686,20 @@ public class RepositoryServiceImpl implements RepositoryService { public Map getCompatibilityClasses(String mode) { logger.debug("Getting compatibility classes for mode: {}", mode); + String lowercaseMode = mode.toLowerCase(); Map retMap = new HashMap<>(); Map compatibilityClasses = this.getVocabulary("dnet:compatibilityLevel").getAsMap(); boolean foundData = false; for (Map.Entry entry : compatibilityClasses.entrySet()) { - if (mode.equalsIgnoreCase(Constants.REPOSITORY_MODE_ALL)) + if (lowercaseMode.equals(Constants.REPOSITORY_MODE_ALL)) return compatibilityClasses; - else if (mode.equalsIgnoreCase(Constants.REPOSITORY_MODE_RE3DATA)) { + else if (lowercaseMode.equals(Constants.REPOSITORY_MODE_RE3DATA)) { if (ValidatorServiceImpl.OPENAIRE_DATA_REGEX.matcher(entry.getKey()).matches()) { retMap.put(entry.getKey(), entry.getValue()); foundData = true; } - } else if (mode.equalsIgnoreCase("cris")) { + } else if (lowercaseMode.startsWith("cris")) { // Future proofing for inconsistent CRIS modes. if (entry.getKey().contains("openaire-cris")) { retMap.put(entry.getKey(), entry.getValue()); foundData = true; @@ -710,7 +711,7 @@ public class RepositoryServiceImpl implements RepositoryService { } //TODO TO BE REMOVED WHEN VOCABULARIES ARE UPDATED - if ((mode.equalsIgnoreCase("repository") || mode.equalsIgnoreCase(Constants.REPOSITORY_MODE_RE3DATA)) && !foundData) + if ((lowercaseMode.equals("repository") || lowercaseMode.equals(Constants.REPOSITORY_MODE_RE3DATA)) && !foundData) retMap.put("openaire2.0_data", "OpenAIRE Data (funded, referenced datasets)"); return retMap; @@ -934,7 +935,12 @@ public class RepositoryServiceImpl implements RepositoryService { private Paging searchApi(UriComponents uriComponents, RequestFilter requestFilter, Class clazz) { Paging repositories = new Paging<>(); ResponseEntity rs; - rs = restTemplate.exchange(uriComponents.toUri(), HttpMethod.GET, new HttpEntity<>(requestFilter), Map.class); + try { + rs = restTemplate.exchange(uriComponents.toUri(), HttpMethod.GET, new HttpEntity<>(requestFilter), Map.class); + } catch (Exception e) { + logger.error("", e); + throw e; + } if (!rs.getStatusCode().is2xxSuccessful()) { logger.error("Api call not successful. Code: {} | Body: {}", rs.getStatusCode(), rs.getBody()); } From 4f1e3a14d4164b4669e0b069939f91dcd62dadbb Mon Sep 17 00:00:00 2001 From: LSmyrnaios Date: Thu, 4 Apr 2024 16:15:17 +0300 Subject: [PATCH 7/7] Fix conflicts in README. --- README.md | 121 ++++++++++++++++++++---------------------------------- 1 file changed, 45 insertions(+), 76 deletions(-) diff --git a/README.md b/README.md index f8fb33e..7dc4421 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,23 @@ -# uoa-repository-manager-service (Provide backend) +# OpenAIRE Provide - backend service ## Introduction +OpenAIRE PROVIDE is the content gateway service of OpenAIRE, where data providers are invited to connect scholarly content with OpenAIRE.
+OpenAIRE PROVIDE allows repositories, data archives, journals, aggregators, CRIS systems, to enter the OpenAIRE and European Open Science (EOSC) ecosystem and be accessible by millions of researchers, research institutes and networks, research funders, policy makers and citizens.
+OpenAIRE PROVIDE lowers any technological barriers, by supporting a series of integrations, therefore, enabling its users to visually access OpenAIRE's services that are responsible for the data harvesting process. +

+There are four distinctive steps from the initial express of interest for OpenAIRE PROVIDE to the actual content availability on OpenAIRE and EOSC. +
+The steps indicate the important subservices of OpenAIRE that perform the following functionalities: +- Validation of data sources with the OpenAIRE guidelines (via the OpenAIRE Validator) +- Registration of data sources to OpenAIRE and global interlinked networks provides links to content for text and data mining, view history of validations, status of harvesting Enrichment of metadata information that describes the data sources to be available through OpenAIRE. +- Subscribe and view/receive notifications to enrich the metadata or the content of the data source (via the OpenAIRE Broker) +- View usage statistics of Open research impact by subscribing to the OpenAIRE UsageCounts service; view aggregated, cleaned usage stats for repository access and broaden your mechanisms for impact assessment. + +
+ ## Architecture +[...] - -
-

## Building @@ -25,8 +37,6 @@ The backend is a [Maven](https://maven.apache.org/index.html) project. It has be 3. Build Maven project
`mvn clean package`
Produces the file "./target/**uoa-repository-manager-service.jar**" which can be run with: - Run the app with: `java -jar ./target/uoa-repository-manager-service.jar` -

-

#### Build using Docker The repository contains a **Dockerfile** which can be used to build an image containing the compiled project. @@ -41,34 +51,9 @@ The repository contains a **Dockerfile** which can be used to build an image con 3. Build Docker image
`docker build . -t ` - -## Deployment - -### Prerequisites -* [PostgreSQL 9.5+](https://www.postgresql.org/) - [...] - - -### Instructions -Execute `java -jar ./target/uoa-repository-manager-service.jar` - - -## Installation - -### Prerequisites -* [PostgreSQL 9.5+](https://www.postgresql.org/) - - -### Infrastructure Installation and Configuration -Install all software found in the list of [Prerequisites](#Prerequisites) using the official documentation. - - -#### PostgreSQL - Configuration -[...] - +
## Configuration - The configuration can be set inside the **src/main/resources/application.yml** file. ### Server-related configuration. @@ -111,7 +96,7 @@ services: dev-machine: 88.197.53.71 # VM-71 ``` -#### Configuration about Authentication and authorization infrastructure (AAI). +### Configuration about Authentication and authorization infrastructure (AAI). ``` aai: baseURL: https://aai.openaire.eu @@ -130,14 +115,7 @@ services: url: ${services.provide.aai.baseURL}/registry/ ``` -#### [...] -``` - adminEmail: XX - analyticsURL: https://analytics.openaire.eu/addsite.php? - baseUrl: ${services.openaireServicesBaseUrl}/openaire -``` - -#### Broker's configuration +### Broker's configuration ``` broker: api: api/ @@ -146,7 +124,7 @@ services: url: https://beta.broker.openaire.eu ``` -#### Client's configuration +### Client's configuration ``` clients: dsm: ${services.provide.baseUrl} @@ -155,7 +133,7 @@ services: usagestats: ${services.openaireServicesBaseUrl}/usagestats ``` -#### Database configuration for Provide. +### Database configuration for Provide. ``` db: driverClassName: org.postgresql.Driver @@ -164,13 +142,13 @@ services: username: dnet ``` -#### Configuration for the IS LookUp service. +### Configuration for the IS LookUp service. ``` iSLookUpService: url: https://dev-openaire.d4science.org:443/is/services/isLookUp ``` -#### Mail-server configuration. +### Mail-server configuration. ``` mail: authenticate: true @@ -184,7 +162,7 @@ services: username: XX ``` -#### Redis's configuration +### Redis's configuration ``` redis: host: vereniki.athenarc.gr @@ -192,10 +170,8 @@ services: port: 6379 ``` -#### [...] +### Usage statistics ``` - topic_types: - url: ${services.openaireServicesBaseUrl}/provision/mvc/vocabularies/dnet:topic_types.json # TODO - Check this! The requested json file does not exist in the DEV-url below) usageStatisticsDiagramsBaseURL: https://beta.openaire.eu/stats3/ usageStatisticsNumbersBaseURL: ${services.openaireServicesBaseUrl}/usagestats/datasources/ usagestats: @@ -204,7 +180,7 @@ services: sushiliteR5Endpoint: ${services.openaireServicesBaseUrl}/usagestats_r5/sushilite/r5/ ``` -#### Validator - related configuration. +### Validator - related configuration. ``` validator: results: @@ -213,37 +189,30 @@ services: url: http://${services.provide.dev-machine}:8080/uoa-validator-service/services/validatorWebService ``` +### Miscellaneous +``` + topic_types: + url: ${services.openaireServicesBaseUrl}/provision/mvc/vocabularies/dnet:topic_types.json + adminEmail: XX + analyticsURL: https://analytics.openaire.eu/addsite.php? + baseUrl: ${services.openaireServicesBaseUrl}/openaire +``` +
-
+ +## Deployment + + +### Prerequisites +* install [PostgreSQL 9.5+](https://www.postgresql.org/) +* Run the app with: `java -jar ./target/uoa-repository-manager-service.jar --spring.config.location=application.yml` +
-## Security -## Maintenance -## Recovery -## References - - -
-
-
- ---- +## misc ### Notes for Swagger-UI: - Access it through this url: http://localhost:8480/uoa-repository-manager-service/swagger-ui/index.html - In order to request data from most endpoints, you have to be a "REGISTERED_USER", otherwise you will get a 403 error code. -- In order to be a registered user, you have to run the [UI-service](https://code-repo.d4science.org/MaDgIK/uoa-repository-manager-ui) , in the same machine, at the same time and login through a browser, using the following url: http://localhost:8480/uoa-repository-manager-service/openid_connect_login - - - -## Introduction -## Architecture -## Bulding -## Deployment -## Installation -## Configuration -## Security -## Maintenance -## Recovery -## References \ No newline at end of file +- In order to be a registered user, you have to run the [UI-service](https://code-repo.d4science.org/MaDgIK/uoa-repository-manager-ui) , in the same machine, at the same time and login through a browser, using the following url: http://localhost:8480/uoa-repository-manager-service/openid_connect_login \ No newline at end of file