13 KiB
Architecture
Service
gCube CMS Service is a gCube REST-like Application developped on top of gCube SmartGears.
It implements the core business logic of gCube CMS Suite, exposing methods for the management of Projects, Use Case Descriptors and available server plugins.
It is built on top of Jersey Framework, it uses a MongoDB Document Store for JSON archiving and Querying. It exploits the infrastructure StorageHub for archiving FileSets and the Information System in order to discover available resources.
Interface
Projects
BASE PATH : ../projects/{ucid}/
Projects interface offers the following methods over a collection (see {ucid} in path)
- CRUD operations (create, deleteById, getById, updateById)
- FileSet Management
- Query operations (powered by Mongo DB capabiliteis) allowing for filtering, pagination, projection.
- STEP invocation
- Project configuration, reporting the generated resources for the collection and how to access them.
FileSet Management
Clients can register Filesets with JSON requests with the following fields :
- fieldName : target field name
- parentPath : the target field's parent
- fieldDefinitionPath : JSONPath of the target field definition in UCD schema
- streams : list of JSON objects representing the fileset's payload and filenames. Payload can be specified either as gCube Storage Hub volatile area ID or as URL
- clashOptions : one of REPLACE_EXISTING, MERGE_EXISTING, APPEND
- toSetAccess [optional]: to set access policy for the fileset. Default is taken from the UCD.
- attributes [optional] : additional attributes to set to the FileSet.
Following example is taken from Concessioni use case submission :
Querying
The service supports querying against Projects collections. Clients can submit query requests as a JSON object with the following fields :
- ordering [optional] : bean with both direction and fields list
- paging [optional]: bean with both offset and limit
- projection [optional] (NB see mongoDB aggregation pipelines): the project fields to be returned
- filtering [optional] (NB see mongoDB aggregation pipelines): the condition to evaluate
Following are examples of queries towards collections (in some of the following example we consider structure and lifecycle from "Concessioni" UCD) :
Query lifecycleInformation ordered by submission
- ::
-
- {
-
"ordering" :{"direction" : "DESCENDING", "fields" :["_info._creationInfo._instant"]}, "projection" :{"_lifecycleInformation._phase":1,"_theDocument.nome":1,"_lifecycleInformation._lastOperationStatus":1,"_lifecycleInformation._errorMessages":1,"_info._creationInfo._instant":1}
}
Query filesets ordered by submission
- ::
-
- {
-
"ordering" :{"direction" : "DESCENDING", "fields" :["_info._creationInfo._instant"]}, "projection" :{"_theDocument.abstractRelazione.fileset._payloads" : 1,"_theDocument.immaginiRappresentative.fileset._payloads" : 1, "_theDocument.pianteFineScavo.fileset._payloads" : 1,"_theDocument.posizionamentoScavo.fileset._payloads" : 1}
}
Query projects with "follow" relationship ordered by name
- ::
-
- {
-
"ordering" :{"direction" : "ASCENDING", "fields" :["_theDocument.nome"]}, "filter" : {"_relationships._relationshipName" : {"$eq":"follows"}}
}
STEP Invocation
Lifecycle STEPs can be invoked by submitting JSON requests with the following fields :
- stepID : the ID of the STEP to be executed.
- options : the parameters needed for the STEP exewcution.
Project configuration
Some resources are generated by the service in the normal execution of Projects lifecycle (e.g. indexes). The service exposes the following informations in order to allow clients (e.g. GUIs) to properly consume these resources.
- indexes : A set of generated indexes (e.g. SDI centroid layers)
- archives : A set of generated archive (e.g. Document archive) along with related information.
- errorMessages : A set of error messages reporting errors in generating the current response
- warningMessages : A set of warning messages reporting unexpected behaviour in generating the current response
- status : the global status.
Following is an example of the returned information :: { "profile_id": "profiledConcessioni", "context": "/pred4s/preprod/preVRE", "last_updated_time": "2022-11-17T14:58:59.026", "indexes": [ {"_type": "GIS-CENTROIDS", "layer": {"_type": "gcube-sdi-layer", "_platformInfo": [{ "_type": "Geoserver", "workspace": "profiledconcessioni_prevre", "storeName": "profiledconcessioni_prevre_centroids", "_host": "geoserver1-t.pre.d4science.org", "layerName": "profiledconcessioni_prevre_centroids" }], "_bbox": {"_maxX": 180.0,"_minX": -180.0,"_maxY": 90.0,"_minY": -90.0}, "_ogcLinks": { "wms": "https://geoserver1-t.pre.d4science.org/geoserver/profiledconcessioni_prevre/wms?service=WMS&version=1.1.0&request=GetMap&layers=profiledconcessioni_prevre:profiledconcessioni_prevre_centroids&styles=&bbox=-180.000000,-90.000000,180.000000,90.000000&srs=EPSG:4326&format=application/openlayers&width=400&height=400" } }, "indexName": "profiledconcessioni_prevre_centroids", "records": 1, "crossReferencedLayers": {}, "flag": "public" }], "archives": [ { "_type": "DOCUMENT-STORE-COLLECTION", "count": 225, "collection_name": "profiledConcessioni", "countByPhase": [ {"_id": {"phase": "DRAFT","status": "OK"},"count": 19}, {"_id": {"phase": "PUBLISHED","status": "OK"},"count": 3}, {"_id": {"phase": "Pending Approval","status": "ERROR"},"count": 11}, ... ] }, { "_type": "W-STORAGE", "folder_id": "ded73475-f9b1-46c8-94f8-5b35c60667ce" } ], "errorMessages": null, "warningMessages": null, "status": "OK" }
UCD Management
Current approach uses the class org.gcube.application.geoportal.service.engine.providers.ucd.SingleISResourceUCDProvider as a UCD Provider.
This class asks the gCube Information System a Generic Resource with the following coordinates :
- SecondaryType : CMS
- Name : UCDs
It expects the body to be like the following example :: <UCDs> <record label="Human readable name" ucdUrl="some url" defaultForRegex="[opt] regex"/> <record label="Another readable name" ucdUrl="some other url" defaultForRegex="[opt] regex"/> </UCDs>
Project Management
Projects metadata are stored as JSON document in a Document Store DB (powered by MongoDB) called Project archive. The core logic of Project management lies in the class org.gcube.application.geoportal.service.engine.mongo.ProfiledMongoManager, which is the only component in charge of accessing the Project archive, in order to grant robustness via isolation.
Locking
ProfiledMongoManager class implements the following lock mechanism for all writing executions (here represented for the STEP execution scenario for completeness).
Plugin Framework
In order to isolate logic, Plugins are discovered via reflection. Plugins are thus expected to implement the proper interface, choosing between the ones defined in the framework and listed below.
The framework provides also : * Definition of all involved Requests, Response and Faults beans * Basic plugin implementations for common use cases, that can be extended for custom solutions (e.g. AbstractLifeCycleManager, SingleLifecycleManager, Default3PhaseManager) * Common utilities for plugin discovery (e.g. for Lifecycle Manager to invoke Materializers and Indexers) * Engine Provider in order to allow plugins to access implementations provided by the service * Common utilities (e.g. serialization)
Engine Provider
org.gcube.application.cms.implementations.ImplementationProvider class represents a common point of implementations reference for both the service and plugins.
It exposes registered implementations by asking for an Engine (generic interface org.gcube.application.cms.caches.Engine<T>) responsible to provide a specific object class.
The service (or a startup environment, e.g. for testing purposes) is expected to register the implemented Engines at startup.
In the following list there are all Engine<T> implementations provided by the service and available to Plugins :
- MongoClientProvider provided class T : org.gcube.application.geoportal.service.model.internal.db.Mongo
- ProjectAccessProvider provided class T : org.gcube.application.cms.implementations.ProjectAccess
- StorageClientProvider provided class T : org.gcube.application.geoportal.common.utils.StorageUtils
- SingleISResourceUCDProviderp rovided class T : org.gcube.application.geoportal.service.engine.providers.ucd.ProfileMap
- PluginManager provided class T : org.gcube.application.geoportal.service.engine.providers.PluginManager.PluginMap
- UCDManager provided class T : org.gcube.application.geoportal.service.engine.mongo.UCDManagerI
- ConfigurationCacheprovided class T : org.gcube.application.geoportal.service.engine.providers.ConfigurationCache.ConfigurationMap
Plugin Interfaces
The following is a list of defined plugin interfaces, which implementations are expected to implement in order to make themself available.
Following diagram shows the interfaces defined in the framework (package org.gcube.application.cms.plugins)