@ -39,15 +39,15 @@ It exploits the infrastructure StorageHub for archiving FileSets and the Informa
Interface
=========
..note:: The service interface is available for testing here.
.. note :: The service interface is available for testing here.
..alert:: You need gCube credentials in order to send requests to the service.
.. alert :: You need gCube credentials in order to send requests to the service.
Projects
********
BASE PATH : ''../projects/{ucid}/''
BASE PATH : **../projects/{ucid}/**
Projects interface offers the following methods over a collection (see {ucid} in path)
@ -57,73 +57,237 @@ Projects interface offers the following methods over a collection (see {ucid} in
* STEP invocation
* Project configuration, reporting the generated resources for the collection and how to access them.
FileSet Management
==================
Clients can register Filesets with JSON requests with the following fields :
* **fieldName** : target field name
* **parentPath** : the target field's parent
* **fieldDefinitionPath** : JSONPath of the target field definition in UCD schema
* **streams** : list of JSON objects representing the fileset's payload and filenames. Payload can be specified either as **gCube Storage Hub volatile area ID** or as **URL**
* **clashOptions** : one of **REPLACE_EXISTING**, **MERGE_EXISTING**, **APPEND**
* **toSetAccess** [optional]: to set access policy for the fileset. Default is taken from the UCD.
* **attributes** [optional] : additional attributes to set to the FileSet.
Following example is taken from *Concessioni* use case submission :
Querying
========
The service supports querying against Projects collections. Clients can submit query requests as a JSON object with the following fields :
* **ordering** [optional] : bean with both **direction** and **fields** list
* **paging** [optional]: bean with both **offset** and **limit**
* **projection** [optional] (NB see mongoDB aggregation pipelines): the project fields to be returned
* **filtering** [optional] (NB see mongoDB aggregation pipelines): the condition to evaluate
Following are examples of queries towards collections (in some of the following example we consider structure and lifecycle from *"Concessioni"* UCD) :
Query **lifecycleInformation** ordered by submission
Lifecycle STEPs can be invoked by submitting JSON requests with the following fields :
* **stepID** : the ID of the STEP to be executed.
* **options** : the parameters needed for the STEP exewcution.
.. alert :: STEP ID needs to be the same expected by the configured LifeCycle Manager
.. note :: You can check the supported STEPs along with its parameters, just by interacting with the **Plugins** interface described below.
Project configuration
=====================
Some resources are generated by the service in the normal execution of Projects lifecycle (e.g. indexes).
The service exposes the following informations in order to allow clients (e.g. GUIs) to properly consume these resources.
.. alert :: Format of the resource representation vary according to their nature in order to adapt to the needs of the collection's lifecycle.
* **indexes** : A set of generated indexes (e.g. SDI centroid layers)
* **archives** : A set of generated archive (e.g. Document archive) along with related information.
* **errorMessages** : A set of error messages reporting errors in generating the current response
* **warningMessages** : A set of warning messages reporting unexpected behaviour in generating the current response
* **status** : the global status.
.. note :: In the current implementation all collection uses the same **DOCUMENT-STORE-COLLECTION** (based on MongoDB) and **W-STORAGE** (based on gCube StorageHub).
.. note :: **DOCUMENT-STORE-COLLECTION** archive also reports aggregated Projects count by PHASE and STATUS.
Following is an example of the returned information
<record label="Another readable name" ucdUrl="some other url" defaultForRegex="[opt] regex"/>
</UCDs>
==================
******************
Project Management
==================
******************
Projects metadata are stored as JSON document in a Document Store DB (powered by MongoDB) called Project archive. The core logic of Project management lies in the class *org.gcube.application.geoportal.service.engine.mongo.ProfiledMongoManager*, which is the only component in charge of accessing the Project archive, in order to grant robustness via isolation.
Project archive
***************
.. note :: In order to execute both lifecyle **EVENTs** and **STEPs**, the service locates the configured LifecycleManager handler and passes to them requests with all the required information (check **Plugin Framework** below).
Locking
=======
Fileset management
******************
STEP Execution
**************
*ProfiledMongoManager* class implements the following lock mechanism for all writing executions (here represented for the STEP execution scenario for completeness).
.. figure :: _static/imgs/step_execution.png
:alt:Projects locking mechanism
****************
Plugin Framework
****************
In order to isolate logic, Plugins are discovered via reflection.
Plugins are thus expected to implement the proper interface, choosing between the ones defined in the framework and listed below.
.. alert :: Plugins **SHOULD** depend **only** by the Plugin Framework and their specific dependencies.
The framework provides also :
* Definition of all involved **Requests**, **Response** and **Faults** beans
* Basic plugin implementations for common use cases, that can be extended for custom solutions (e.g. AbstractLifeCycleManager, SingleLifecycleManager, Default3PhaseManager)
* Common utilities for plugin discovery (e.g. for Lifecycle Manager to invoke Materializers and Indexers)
* **Engine Provider** in order to allow plugins to access implementations provided by the service
* Common utilities (e.g. serialization)
===============
Engine Provider
===============
*org.gcube.application.cms.implementations.ImplementationProvider* class represents a common point of implementations reference for both the service and plugins.
It exposes registered implementations by asking for an Engine (generic interface *org.gcube.application.cms.caches.Engine*<T>) responsible to provide a specific object class.
The service (or a startup environment, e.g. for testing purposes) is expected to register the implemented Engines at startup.
.. note ::Some of the provided engines also implements **Cache** (generic cache) and **AbstractScopedMap** (per context cache).
In the following list there are all **Engine<T>** implementations provided by the service and available to Plugins :
* MongoClientProvider provided class T : org.gcube.application.geoportal.service.model.internal.db.Mongo
* ProjectAccessProvider provided class T : org.gcube.application.cms.implementations.ProjectAccess
* StorageClientProvider provided class T : org.gcube.application.geoportal.common.utils.StorageUtils
* SingleISResourceUCDProviderp rovided class T : org.gcube.application.geoportal.service.engine.providers.ucd.ProfileMap
* PluginManager provided class T : org.gcube.application.geoportal.service.engine.providers.PluginManager.PluginMap
* UCDManager provided class T : org.gcube.application.geoportal.service.engine.mongo.UCDManagerI
* ConfigurationCacheprovided class T : org.gcube.application.geoportal.service.engine.providers.ConfigurationCache.ConfigurationMap
=================
Plugin Interfaces
=================
The following is a list of defined plugin interfaces, which implementations are expected to implement in order to make themself available.
Following diagram shows the interfaces defined in the framework (package *org.gcube.application.cms.plugins*)
.. figure :: _static/imgs/plugin_interfaces.png
:alt:Plugin interfaces
.. note :: **Plugin** interface as root of the above hierarchy grants common behaviour to all plugins, i.e. the exposition of PluginDescriptor object, which information are returned by the service Plugin REST interface.
Fabio Sinibaldi