153 lines
10 KiB
HTML
153 lines
10 KiB
HTML
<!DOCTYPE html>
|
||
<html class="writer-html5" lang="en" >
|
||
<head>
|
||
<meta charset="utf-8" /><meta name="generator" content="Docutils 0.18.1: http://docutils.sourceforge.net/" />
|
||
|
||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||
<title>Introduction — CCP 0.1.0 documentation</title>
|
||
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
|
||
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
|
||
<link rel="canonical" href="/docs/introduction.html" />
|
||
<!--[if lt IE 9]>
|
||
<script src="_static/js/html5shiv.min.js"></script>
|
||
<![endif]-->
|
||
|
||
<script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
|
||
<script src="_static/jquery.js"></script>
|
||
<script src="_static/underscore.js"></script>
|
||
<script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
|
||
<script src="_static/doctools.js"></script>
|
||
<script src="_static/sphinx_highlight.js"></script>
|
||
<script src="_static/js/theme.js"></script>
|
||
<link rel="index" title="Index" href="genindex.html" />
|
||
<link rel="search" title="Search" href="search.html" />
|
||
<link rel="next" title="User manual" href="usermanual/index.html" />
|
||
<link rel="prev" title="Welcome to CCP’s documentation!" href="index.html" />
|
||
</head>
|
||
|
||
<body class="wy-body-for-nav">
|
||
<div class="wy-grid-for-nav">
|
||
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
|
||
<div class="wy-side-scroll">
|
||
<div class="wy-side-nav-search" >
|
||
|
||
|
||
|
||
<a href="index.html" class="icon icon-home">
|
||
CCP
|
||
</a>
|
||
<div role="search">
|
||
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
|
||
<input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
|
||
<input type="hidden" name="check_keywords" value="yes" />
|
||
<input type="hidden" name="area" value="default" />
|
||
</form>
|
||
</div>
|
||
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
|
||
<p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
|
||
<ul class="current">
|
||
<li class="toctree-l1 current"><a class="current reference internal" href="#">Introduction</a><ul>
|
||
<li class="toctree-l2"><a class="reference internal" href="#about">About</a></li>
|
||
<li class="toctree-l2"><a class="reference internal" href="#architecture">Architecture</a></li>
|
||
</ul>
|
||
</li>
|
||
<li class="toctree-l1"><a class="reference internal" href="usermanual/index.html">User manual</a></li>
|
||
<li class="toctree-l1"><a class="reference internal" href="developermanual/index.html">Developer manual</a></li>
|
||
</ul>
|
||
|
||
</div>
|
||
</div>
|
||
</nav>
|
||
|
||
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
|
||
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
|
||
<a href="index.html">CCP</a>
|
||
</nav>
|
||
|
||
<div class="wy-nav-content">
|
||
<div class="rst-content">
|
||
<div role="navigation" aria-label="Page navigation">
|
||
<ul class="wy-breadcrumbs">
|
||
<li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
|
||
<li class="breadcrumb-item active">Introduction</li>
|
||
<li class="wy-breadcrumbs-aside">
|
||
<a href="_sources/introduction.rst.txt" rel="nofollow"> View page source</a>
|
||
</li>
|
||
</ul>
|
||
<hr/>
|
||
</div>
|
||
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
|
||
<div itemprop="articleBody">
|
||
|
||
<section id="introduction">
|
||
<h1>Introduction<a class="headerlink" href="#introduction" title="Permalink to this heading"></a></h1>
|
||
<section id="about">
|
||
<h2>About<a class="headerlink" href="#about" title="Permalink to this heading"></a></h2>
|
||
<p>CCP is the D4Science Cloud Computing Platform built upon the experience of the previous Dataminer initiative <<a class="reference external" href="https://en.wikipedia.org/wiki/D4Science">https://en.wikipedia.org/wiki/D4Science</a>> and uses a novel approach based on containerization, REST APIs and Json.</p>
|
||
<p>Several fields of ICT have experienced a major evolution during the last decade and many new advances, such as the widespread adoption of microservice development patterns. This resulted in substantial improvements in terms of interoperability and composability of software artefacts.
|
||
The vast landscape of new opportunities, in addition to the greatly increased requirements and expectations, have been the drivers for the design and development of a new Cloud Computing Platform that represents the result of a global rethink of the Data Miner.</p>
|
||
</section>
|
||
<section id="architecture">
|
||
<h2>Architecture<a class="headerlink" href="#architecture" title="Permalink to this heading"></a></h2>
|
||
<p>A logical vision of the CCP architecutre is depicted in the following Figure.</p>
|
||
<figure class="align-default" id="id1">
|
||
<img alt="Logical vision of architecture" src="_images/logicalvisionarchitecture.png" />
|
||
<figcaption>
|
||
<p><span class="caption-text">The CCP logical vision of architecture</span><a class="headerlink" href="#id1" title="Permalink to this image"></a></p>
|
||
</figcaption>
|
||
</figure>
|
||
<p>In this vision, CCP is a layered set of components starting at the bottom with the <strong>Infrastructure</strong> layer, encompassing components such as hardware, Virtual Machines, container based clusters, storage facilities and networks.</p>
|
||
<p>The <strong>Runtimes</strong> layer offers a set of prebuilt, preconfigured execution environments such as containers or Virtual Machine images.</p>
|
||
<p>The <strong>Method</strong> layer contains specification of computational methods that can be anything, from social mining algorithms to AI classifiers and data harvesters. Data scientists with development skills are encouraged to develop new Methods or cloning existing ones, being their responsibility to choose compatible runtimes or propose new ones to be integrated. Tools for sharing the Methods with communities such as Virtual Research Environments are made available at this layer.</p>
|
||
<p>The overall user community works at the <strong>Workbench</strong> layer, which is the abstraction of overarching tools that are able to directly use the available Methods, compose them into workflows and integrate them into visual tools, such as Jupyter Notebook, R scripts, Knime workflows.</p>
|
||
<p><strong>Experimentation</strong> is the term that defines the activity of configuring new Runtimes, defining new Methods and using them in the Workbench.</p>
|
||
<p>In the opposite direction, <strong>Consolidation</strong> represents the possibility to transform dynamic objects into more static ones in order to improve reusability, portability and overall performance. For instance, workflows or combinations of Methods could be transformed into Methods themselves or even Methods into Runtimes.</p>
|
||
<p>The logical architecture presented in Figure 9 shows the natively distributed nature of CCP.</p>
|
||
<figure class="align-default" id="id2">
|
||
<img alt="Logical architecture" src="_images/logicalarchitecture.png" />
|
||
<figcaption>
|
||
<p><span class="caption-text">The CCP logical architecture</span><a class="headerlink" href="#id2" title="Permalink to this image"></a></p>
|
||
</figcaption>
|
||
</figure>
|
||
<p>Starting from the top, <strong>Infrastructures</strong> (as computing resources that will host CCP method executions, i.e. anything from simple laptops up to clusters of server Hosts) can be connected as runtime execution environments by installing a Controller component. Within an infrastructure, Hosts are computational nodes like physical or virtual servers. They are delegated to execute methods.</p>
|
||
<p><strong>Controllers</strong> are processes that communicate through a specific API with the CCP in order to poll for tasks to perform on the Infrastructure they control. Tasks may include deploying and running methods, cleaning up executions, reporting on the overall status of the Infrastructure.</p>
|
||
<p>In order to keep the current state for CCP, a couple of registries are involved. Specifically, the Method & Execution Registry and the Infrastructure and Runtime registry.</p>
|
||
<p>User driven visual components are available to manage Infrastructures, Runtimes, Methods and Executions at the frontend. Those components are identified by the user icon in the previous Figure.</p>
|
||
<p>Because many of the operations involved are lengthy and asynchronous, CCP includes a Logging Service that is used to send back realtime notifications about the state of a particular process to the user. These notifications include advancement of Executions, advertisement of Infrastructures status updates, and error conditions.</p>
|
||
<p>All complex processes involved in CCP are implemented as workflows inside a Workflow Orchestrator which, in addition to granting a high level of flexibility and customisation, allows for a centralised endpoint to monitor progress and check for errors that may occur.</p>
|
||
<p>At the basis of all interactions among external actors, such as users and Controllers, a strong authentication and authorisation mechanism is enforced by an Identity and Access Management software (IAM). This enables it to address security requirements as well as to implement ownership attribution and auditing.</p>
|
||
</section>
|
||
</section>
|
||
|
||
|
||
</div>
|
||
</div>
|
||
<footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
|
||
<a href="index.html" class="btn btn-neutral float-left" title="Welcome to CCP’s documentation!" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
|
||
<a href="usermanual/index.html" class="btn btn-neutral float-right" title="User manual" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
|
||
</div>
|
||
|
||
<hr/>
|
||
|
||
<div role="contentinfo">
|
||
<p>© Copyright 2023, Marco Lettere.</p>
|
||
</div>
|
||
|
||
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
|
||
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
|
||
provided by <a href="https://readthedocs.org">Read the Docs</a>.
|
||
|
||
|
||
</footer>
|
||
</div>
|
||
</div>
|
||
</section>
|
||
</div>
|
||
<script>
|
||
jQuery(function () {
|
||
SphinxRtdTheme.Navigation.enable(true);
|
||
});
|
||
</script>
|
||
|
||
</body>
|
||
</html> |