Documentation
This commit is contained in:
parent
b1743a09c8
commit
b1515d1c33
|
|
@ -4,7 +4,7 @@
|
|||
Architecture
|
||||
###############
|
||||
|
||||
.. figure:: _static/imgs/architecture.png
|
||||
.. figure :: _static/imgs/architecture.png
|
||||
:alt: CMS Suite overall concept
|
||||
|
||||
gCube CMS suite : General Architecture
|
||||
|
|
@ -39,15 +39,15 @@ It exploits the infrastructure StorageHub for archiving FileSets and the Informa
|
|||
Interface
|
||||
=========
|
||||
|
||||
.. note:: The service interface is available for testing here.
|
||||
.. note :: The service interface is available for testing here.
|
||||
|
||||
.. alert:: You need gCube credentials in order to send requests to the service.
|
||||
.. alert :: You need gCube credentials in order to send requests to the service.
|
||||
|
||||
|
||||
Projects
|
||||
********
|
||||
|
||||
BASE PATH : ''../projects/{ucid}/''
|
||||
BASE PATH : **../projects/{ucid}/**
|
||||
|
||||
Projects interface offers the following methods over a collection (see {ucid} in path)
|
||||
|
||||
|
|
@ -57,73 +57,237 @@ Projects interface offers the following methods over a collection (see {ucid} in
|
|||
* STEP invocation
|
||||
* Project configuration, reporting the generated resources for the collection and how to access them.
|
||||
|
||||
|
||||
FileSet Management
|
||||
==================
|
||||
|
||||
Clients can register Filesets with JSON requests with the following fields :
|
||||
|
||||
* **fieldName** : target field name
|
||||
* **parentPath** : the target field's parent
|
||||
* **fieldDefinitionPath** : JSONPath of the target field definition in UCD schema
|
||||
* **streams** : list of JSON objects representing the fileset's payload and filenames. Payload can be specified either as **gCube Storage Hub volatile area ID** or as **URL**
|
||||
* **clashOptions** : one of **REPLACE_EXISTING**, **MERGE_EXISTING**, **APPEND**
|
||||
* **toSetAccess** [optional]: to set access policy for the fileset. Default is taken from the UCD.
|
||||
* **attributes** [optional] : additional attributes to set to the FileSet.
|
||||
|
||||
Following example is taken from *Concessioni* use case submission :
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Querying
|
||||
========
|
||||
|
||||
The service supports querying against Projects collections. Clients can submit query requests as a JSON object with the following fields :
|
||||
|
||||
* **ordering** [optional] : bean with both **direction** and **fields** list
|
||||
* **paging** [optional]: bean with both **offset** and **limit**
|
||||
* **projection** [optional] (NB see mongoDB aggregation pipelines): the project fields to be returned
|
||||
* **filtering** [optional] (NB see mongoDB aggregation pipelines): the condition to evaluate
|
||||
|
||||
|
||||
Following are examples of queries towards collections (in some of the following example we consider structure and lifecycle from *"Concessioni"* UCD) :
|
||||
|
||||
Query **lifecycleInformation** ordered by submission
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
{
|
||||
"ordering" :{"direction" : "DESCENDING", "fields" :["_info._creationInfo._instant"]},
|
||||
"projection" :{"_lifecycleInformation._phase":1,"_theDocument.nome":1,"_lifecycleInformation._lastOperationStatus":1,"_lifecycleInformation._errorMessages":1,"_info._creationInfo._instant":1}
|
||||
}
|
||||
|
||||
Query **filesets** ordered by submission
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
{
|
||||
"ordering" :{"direction" : "DESCENDING", "fields" :["_info._creationInfo._instant"]},
|
||||
"projection" :{"_theDocument.abstractRelazione.fileset._payloads" : 1,"_theDocument.immaginiRappresentative.fileset._payloads" : 1,
|
||||
"_theDocument.pianteFineScavo.fileset._payloads" : 1,"_theDocument.posizionamentoScavo.fileset._payloads" : 1}
|
||||
}
|
||||
|
||||
Query **projects** with "follow" relationship ordered by *name*
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
{
|
||||
"ordering" :{"direction" : "ASCENDING", "fields" :["_theDocument.nome"]},
|
||||
"filter" : {"_relationships._relationshipName" : {"$eq":"follows"}}
|
||||
}
|
||||
|
||||
|
||||
STEP Invocation
|
||||
===============
|
||||
|
||||
Lifecycle STEPs can be invoked by submitting JSON requests with the following fields :
|
||||
|
||||
* **stepID** : the ID of the STEP to be executed.
|
||||
* **options** : the parameters needed for the STEP exewcution.
|
||||
|
||||
.. alert :: STEP ID needs to be the same expected by the configured LifeCycle Manager
|
||||
|
||||
.. note :: You can check the supported STEPs along with its parameters, just by interacting with the **Plugins** interface described below.
|
||||
|
||||
|
||||
Project configuration
|
||||
=====================
|
||||
|
||||
|
||||
UCDs
|
||||
****
|
||||
|
||||
Capabilities
|
||||
************
|
||||
Some resources are generated by the service in the normal execution of Projects lifecycle (e.g. indexes).
|
||||
The service exposes the following informations in order to allow clients (e.g. GUIs) to properly consume these resources.
|
||||
|
||||
|
||||
.. alert :: Format of the resource representation vary according to their nature in order to adapt to the needs of the collection's lifecycle.
|
||||
|
||||
* **indexes** : A set of generated indexes (e.g. SDI centroid layers)
|
||||
* **archives** : A set of generated archive (e.g. Document archive) along with related information.
|
||||
* **errorMessages** : A set of error messages reporting errors in generating the current response
|
||||
* **warningMessages** : A set of warning messages reporting unexpected behaviour in generating the current response
|
||||
* **status** : the global status.
|
||||
|
||||
==============
|
||||
.. note :: In the current implementation all collection uses the same **DOCUMENT-STORE-COLLECTION** (based on MongoDB) and **W-STORAGE** (based on gCube StorageHub).
|
||||
|
||||
.. note :: **DOCUMENT-STORE-COLLECTION** archive also reports aggregated Projects count by PHASE and STATUS.
|
||||
|
||||
Following is an example of the returned information
|
||||
::
|
||||
{
|
||||
"profile_id": "profiledConcessioni",
|
||||
"context": "/pred4s/preprod/preVRE",
|
||||
"last_updated_time": "2022-11-17T14:58:59.026",
|
||||
"indexes": [
|
||||
{"_type": "GIS-CENTROIDS",
|
||||
"layer": {"_type": "gcube-sdi-layer",
|
||||
"_platformInfo": [{
|
||||
"_type": "Geoserver",
|
||||
"workspace": "profiledconcessioni_prevre",
|
||||
"storeName": "profiledconcessioni_prevre_centroids",
|
||||
"_host": "geoserver1-t.pre.d4science.org",
|
||||
"layerName": "profiledconcessioni_prevre_centroids"
|
||||
}],
|
||||
"_bbox": {"_maxX": 180.0,"_minX": -180.0,"_maxY": 90.0,"_minY": -90.0},
|
||||
"_ogcLinks": {
|
||||
"wms": "https://geoserver1-t.pre.d4science.org/geoserver/profiledconcessioni_prevre/wms?service=WMS&version=1.1.0&request=GetMap&layers=profiledconcessioni_prevre:profiledconcessioni_prevre_centroids&styles=&bbox=-180.000000,-90.000000,180.000000,90.000000&srs=EPSG:4326&format=application/openlayers&width=400&height=400"
|
||||
}
|
||||
},
|
||||
"indexName": "profiledconcessioni_prevre_centroids",
|
||||
"records": 1,
|
||||
"crossReferencedLayers": {},
|
||||
"flag": "public"
|
||||
}],
|
||||
"archives": [
|
||||
{
|
||||
"_type": "DOCUMENT-STORE-COLLECTION",
|
||||
"count": 225,
|
||||
"collection_name": "profiledConcessioni",
|
||||
"countByPhase": [
|
||||
{"_id": {"phase": "DRAFT","status": "OK"},"count": 19},
|
||||
{"_id": {"phase": "PUBLISHED","status": "OK"},"count": 3},
|
||||
{"_id": {"phase": "Pending Approval","status": "ERROR"},"count": 11},
|
||||
...
|
||||
]
|
||||
},
|
||||
{
|
||||
"_type": "W-STORAGE",
|
||||
"folder_id": "ded73475-f9b1-46c8-94f8-5b35c60667ce"
|
||||
}
|
||||
],
|
||||
"errorMessages": null,
|
||||
"warningMessages": null,
|
||||
"status": "OK"
|
||||
}
|
||||
|
||||
**************
|
||||
UCD Management
|
||||
==============
|
||||
**************
|
||||
|
||||
Current approach uses the class *org.gcube.application.geoportal.service.engine.providers.ucd.SingleISResourceUCDProvider* as a UCD Provider.
|
||||
|
||||
This class asks the *gCube Information System* a **Generic Resource** with the following coordinates :
|
||||
|
||||
* SecondaryType : **CMS**
|
||||
* Name : **UCDs**
|
||||
|
||||
It expects the body to be like the following example
|
||||
::
|
||||
<UCDs>
|
||||
<record label="Human readable name" ucdUrl="some url" defaultForRegex="[opt] regex"/>
|
||||
<record label="Another readable name" ucdUrl="some other url" defaultForRegex="[opt] regex"/>
|
||||
</UCDs>
|
||||
|
||||
|
||||
==================
|
||||
******************
|
||||
Project Management
|
||||
==================
|
||||
******************
|
||||
|
||||
Projects metadata are stored as JSON document in a Document Store DB (powered by MongoDB) called Project archive. The core logic of Project management lies in the class *org.gcube.application.geoportal.service.engine.mongo.ProfiledMongoManager*, which is the only component in charge of accessing the Project archive, in order to grant robustness via isolation.
|
||||
|
||||
Project archive
|
||||
***************
|
||||
.. note :: In order to execute both lifecyle **EVENTs** and **STEPs**, the service locates the configured LifecycleManager handler and passes to them requests with all the required information (check **Plugin Framework** below).
|
||||
|
||||
Locking
|
||||
=======
|
||||
|
||||
|
||||
Fileset management
|
||||
******************
|
||||
|
||||
|
||||
STEP Execution
|
||||
**************
|
||||
*ProfiledMongoManager* class implements the following lock mechanism for all writing executions (here represented for the STEP execution scenario for completeness).
|
||||
|
||||
|
||||
.. figure :: _static/imgs/step_execution.png
|
||||
:alt: Projects locking mechanism
|
||||
|
||||
****************
|
||||
Plugin Framework
|
||||
****************
|
||||
|
||||
In order to isolate logic, Plugins are discovered via reflection.
|
||||
Plugins are thus expected to implement the proper interface, choosing between the ones defined in the framework and listed below.
|
||||
|
||||
.. alert :: Plugins **SHOULD** depend **only** by the Plugin Framework and their specific dependencies.
|
||||
|
||||
The framework provides also :
|
||||
* Definition of all involved **Requests**, **Response** and **Faults** beans
|
||||
* Basic plugin implementations for common use cases, that can be extended for custom solutions (e.g. AbstractLifeCycleManager, SingleLifecycleManager, Default3PhaseManager)
|
||||
* Common utilities for plugin discovery (e.g. for Lifecycle Manager to invoke Materializers and Indexers)
|
||||
* **Engine Provider** in order to allow plugins to access implementations provided by the service
|
||||
* Common utilities (e.g. serialization)
|
||||
|
||||
===============
|
||||
Engine Provider
|
||||
===============
|
||||
|
||||
*org.gcube.application.cms.implementations.ImplementationProvider* class represents a common point of implementations reference for both the service and plugins.
|
||||
|
||||
It exposes registered implementations by asking for an Engine (generic interface *org.gcube.application.cms.caches.Engine*<T>) responsible to provide a specific object class.
|
||||
|
||||
The service (or a startup environment, e.g. for testing purposes) is expected to register the implemented Engines at startup.
|
||||
|
||||
.. note ::Some of the provided engines also implements **Cache** (generic cache) and **AbstractScopedMap** (per context cache).
|
||||
|
||||
|
||||
In the following list there are all **Engine<T>** implementations provided by the service and available to Plugins :
|
||||
|
||||
* MongoClientProvider provided class T : org.gcube.application.geoportal.service.model.internal.db.Mongo
|
||||
* ProjectAccessProvider provided class T : org.gcube.application.cms.implementations.ProjectAccess
|
||||
* StorageClientProvider provided class T : org.gcube.application.geoportal.common.utils.StorageUtils
|
||||
* SingleISResourceUCDProviderp rovided class T : org.gcube.application.geoportal.service.engine.providers.ucd.ProfileMap
|
||||
* PluginManager provided class T : org.gcube.application.geoportal.service.engine.providers.PluginManager.PluginMap
|
||||
* UCDManager provided class T : org.gcube.application.geoportal.service.engine.mongo.UCDManagerI
|
||||
* ConfigurationCacheprovided class T : org.gcube.application.geoportal.service.engine.providers.ConfigurationCache.ConfigurationMap
|
||||
|
||||
|
||||
=================
|
||||
Plugin Interfaces
|
||||
=================
|
||||
|
||||
The following is a list of defined plugin interfaces, which implementations are expected to implement in order to make themself available.
|
||||
|
||||
|
||||
Following diagram shows the interfaces defined in the framework (package *org.gcube.application.cms.plugins*)
|
||||
|
||||
.. figure :: _static/imgs/plugin_interfaces.png
|
||||
:alt: Plugin interfaces
|
||||
|
||||
|
||||
.. note :: **Plugin** interface as root of the above hierarchy grants common behaviour to all plugins, i.e. the exposition of PluginDescriptor object, which information are returned by the service Plugin REST interface.
|
||||
|
||||
|
||||
==========================
|
||||
Plugin Development support
|
||||
==========================
|
||||
|
|
|
|||
Loading…
Reference in New Issue