updated source

source/architecture.md

@@ -1,53 +1,27 @@
-Introduction
-============
-
-About
------
-
-Cloud Computing Platform (CCP) is a type of service that offers on-demand and scalable availability of computing resources such as servers, storage, and software over the internet. It eliminates the need for the users to manage their own physical infrastructure, while being able to execute, reexecute, demonstrate and share their procedures, algorithms, comuptations and results with their scientific community independently technological and implementation details. 
-
-CCP has many benefits, such as:
- * Faster time to science, allowing developers to accelerate development with quick deployments
- * Scalability and flexibility, giving more flexibility to adjust computing resources according to the users needs
- * Better collaboration, enabling teams to work together from anywhere with an internet connection and access the same data and applications
- * Productivity, allowing users to focus on their core science functions and innovation without worrying about the underlying infrastructure or  maintenance
- * Performance, delivering high-speed and reliable services with minimal downtime or latency
- * Disaster recovery, enabling users to restore methods and applications quickly and easily in the event of a crisis
- * Competitiveness, giving users access to the most innovative and cutting-edge technology available and helping them stay ahead of the market trends
-
-CCP is built upon the experience of the previous Dataminer initiative <https://en.wikipedia.org/wiki/D4Science> and uses a novel approach based on containerization, REST APIs and Json.
-
-Several fields of ICT have experienced a major evolution during the last decade and many new advances, such as the widespread adoption of microservice development patterns. This resulted in substantial improvements in terms of interoperability and composability of software artefacts.
-The vast landscape of new opportunities, in addition to the greatly increased requirements and expectations, have been the drivers for the design and development of a new Cloud Computing Platform that represents the result of a global rethink of the Data Miner.
-
-Architecture
-------------
+# Architecture
 
+## CCP logical vision of architecture
 A logical vision of the CCP architecutre is depicted in the following Figure.
 
-.. figure:: images/logicalvisionarchitecture.png
-    :alt: Logical vision of architecture
-
-    The CCP logical vision of architecture
+![The CCP logical vision of architecture](images/logicalvisionarchitecture.png)
 
 In this vision, CCP is a layered set of components starting at the bottom with the **Infrastructure** layer, encompassing components such as hardware, Virtual Machines, container based clusters, storage facilities and networks.
 
-The **Runtimes** layer offers a set of prebuilt, preconfigured execution environments such as containers or Virtual Machine images. 
+The **Runtimes** layer offers a set of prebuilt, preconfigured execution environments such as containers or Virtual Machine images.
 
 The **Method** layer contains specification of computational methods that can be anything, from social mining algorithms to AI classifiers and data harvesters. Data scientists with development skills are encouraged to develop new Methods or cloning existing ones, being their responsibility to choose compatible runtimes or propose new ones to be integrated. Tools for sharing the Methods with communities such as Virtual Research Environments are made available at this layer.
 
 The overall user community works at the **Workbench** layer, which is the abstraction of overarching tools that are able to directly use the available Methods, compose them into workflows and integrate them into visual tools, such as Jupyter Notebook, R scripts, Knime workflows.
 
-**Experimentation** is the term that defines the activity of configuring new Runtimes, defining new Methods and using them in the Workbench. 
+**Experimentation** is the term that defines the activity of configuring new Runtimes, defining new Methods and using them in the Workbench.
 
 In the opposite direction, **Consolidation** represents the possibility to transform dynamic objects into more static ones in order to improve reusability, portability and overall performance. For instance, workflows or combinations of Methods could be transformed into Methods themselves or even Methods into Runtimes.
 
+## CCP logical architecture
+
 The logical architecture presented in Figure 9 shows the natively distributed nature of CCP.
 
-.. figure:: images/logicalarchitecture.png
-    :alt: Logical architecture
-
-    The CCP logical architecture
+![The CCP logical architecture](images/logicalarchitecture.png)
 
 Starting from the top, **Infrastructures** (as computing resources that will host CCP method executions, i.e. anything from simple laptops up to clusters of server Hosts) can be connected as runtime execution environments by installing a Controller component. Within an infrastructure, Hosts are computational nodes like physical or virtual servers. They are delegated to execute methods.
 
@@ -61,4 +35,4 @@ Because many of the operations involved are lengthy and asynchronous, CCP includ
 
 All complex processes involved in CCP are implemented as workflows inside a Workflow Orchestrator which, in addition to granting a high level of flexibility and customisation, allows for a centralised endpoint to monitor progress and check for errors that may occur. 
 
-At the basis of all interactions among external actors, such as users and Controllers, a strong authentication and authorisation mechanism is enforced by an Identity and Access Management software (IAM). This enables it to address security requirements as well as to implement ownership attribution and auditing.
+At the basis of all interactions among external actors, such as users and Controllers, a strong authentication and authorisation mechanism is enforced by an Identity and Access Management software (IAM). This enables it to address security requirements as well as to implement ownership attribution and auditing.

source/index.rst

@@ -12,6 +12,8 @@ Welcome to CCP's documentation!
 
    introduction
    quickstart/index
+   methods_development/index
+   architecture
    usermanual/index
    developermanual/index
 

source/introduction.md

@@ -0,0 +1,18 @@
+# Introduction
+
+Cloud Computing Platform (CCP) is a type of service that offers on-demand and scalable availability of computing resources such as servers, storage, and software over the internet. It eliminates the need for the users to manage their own physical infrastructure, while being able to execute, reexecute, demonstrate and share their procedures, algorithms, comuptations and results with their scientific community independently technological and implementation details. 
+
+CCP has many benefits, such as:
+
+* Faster time to science, allowing developers to accelerate development with quick deployments
+* Scalability and flexibility, giving more flexibility to adjust computing resources according to the users needs
+* Better collaboration, enabling teams to work together from anywhere with an internet connection and access the same data and applications
+* Productivity, allowing users to focus on their core science functions and innovation without worrying about the underlying infrastructure or  maintenance
+* Performance, delivering high-speed and reliable services with minimal downtime or latency
+* Disaster recovery, enabling users to restore methods and applications quickly and easily in the event of a crisis
+* Competitiveness, giving users access to the most innovative and cutting-edge technology available and helping them stay ahead of the market trends
+
+CCP is built upon the experience of the previous Dataminer initiative <https://en.wikipedia.org/wiki/D4Science> and uses a novel approach based on containerization, REST APIs and Json.
+
+Several fields of ICT have experienced a major evolution during the last decade and many new advances, such as the widespread adoption of microservice development patterns. This resulted in substantial improvements in terms of interoperability and composability of software artefacts.
+The vast landscape of new opportunities, in addition to the greatly increased requirements and expectations, have been the drivers for the design and development of a new Cloud Computing Platform that represents the result of a global rethink of the Data Miner.

source/methods_development/02.1_basic_docker_image.md

@@ -4,11 +4,11 @@ Simple algorithms can be implemented using a generic unix docker image, and tota
 
 Examples of this algorithms are parametric `curl` or `wget` operations on external APIs, shell for file processing, etc
 
-### Default ccpimage for simple algorithms
+## Default ccpimage for simple algorithms
 
 To implement methods that doesn't need any particular library or configuration, we can leave empty the `ccpimage` value, to use a basic Linux Docker image.
 
-![ccpimage bash](./imgs/input_ccpimage_bash.png)
+![ccpimage bash](../_static/imgs/methods_development/input_ccpimage_bash.png)
 
 ```json
 {
@@ -18,7 +18,7 @@ To implement methods that doesn't need any particular library or configuration,
             "title": "Runtime",
             "description": "`bash`, a basic unix docker image.",
             "minOccurs": 1,
-            "maxOccurs": 1,Bash 
+            "maxOccurs": 1, 
             "schema": {
                 "type": "string",
                 "format": "url",
@@ -31,7 +31,7 @@ To implement methods that doesn't need any particular library or configuration,
 }
 ```
 
-### Execution Parameters
+## Execution Parameters
 
 all the parameters and configurations needed for the execution of the algorithms are defined in the inputs section
 
@@ -118,15 +118,15 @@ and what is parametric (readonly=false, or not defined in the input schema)
 
 the given example uses special input parameters that will be explained in next sections
 
-* [credentials](./credentials.md)
+* [credentials](./03_5_credentials.md)
 
-![secret input credentials](./imgs/input_secret_credentials.png)
+![secret input credentials](../_static/imgs/methods_development/input_secret_credentials.png)
 
-* [file upload](./input_file.md)
+* [file upload](./03_1_input_file.md
 
-![secret file](./imgs/input_file.png)
+![input file](../_static/imgs/methods_development/input_file.png)
 
-### Execution
+## Execution
 
 the implementation of the algorithms is totally defined in the deploy-script and in the execute-script
 ```json
@@ -156,4 +156,3 @@ the implementation of the algorithms is totally defined in the deploy-script and
 
 
 example: [bash example ](../methods_examples/Bash%20Example-1.0.0.json)
-

source/methods_development/02.2_basic_python.md

@@ -1,12 +1,12 @@
-### Running a Python implementation
+# Python algoritms
 
 Python methods can be implemented compiling the *ccpimage* field with a standard python docker image (es `python:3.9.19`)
 
-![python:3.9.19](./imgs/input_ccpimage_python.png)
+![python:3.9.19](../_static/imgs/methods_development/input_ccpimage_python.png)
 
 We can then define an input `repository` field
 
-![input repository](./imgs/input_repository.png)
+![input repository](../_static/imgs/methods_development/input_repository.png)
 
 and configure the  *deploy script* to git clone the repository and install the dependencies, using the placeholder `{{repository}}` to refer the input repository field
 
@@ -39,7 +39,7 @@ then the *run script* will run it passing the required parameters
 }
 ```
 
-### Execution Parameters
+## Execution Parameters
 
 all the parameters and configurations needed for the execution of the algorithms are defined in the input sections
 
@@ -126,12 +126,13 @@ and what is parametric (readonly=false, or not defined in the input schema)
 
 the given example uses special input parameters that will be explained in next sections
 
-* [credentials](./credentials.md)
-* [file upload](./input_file.md)
+* [credentials](./03_5_credentials.md)
+* [file upload](./03_1_input_file.md)
 
-### Execution
+## Execution
 
 the implementation of the algorithms is totally defined in the deploy-script and in the execute-script
+
 ```json
 {
     "additionalParameters": {
@@ -256,4 +257,4 @@ full example
 }
 ```
 
-example: [Gate Cloud Hate And Abuse Detection ](../ccp_methods/Gate%20Cloud%20Toxic%20Language%20Classifier-1.0.0.json)
+example: [WordCloud ](../methods_examples/WordCloud-1.0.0.json)

source/methods_development/03_1_input_file.md

@@ -2,7 +2,7 @@
 
 the input with format "file" allows the user to upload a file while executing the method
 
-![input file](./imgs/input_file.png)
+![input file](../_static/imgs/methods_development/input_file.png)
 
 file size is limited to 100Kb, because the content of the file is base64 encoded and passed as a variable
 
@@ -22,8 +22,7 @@ file size is limited to 100Kb, because the content of the file is base64 encoded
                 "default": ""
             }
         }
-    },
-    ...
+    }
 }
 ```
 
@@ -33,12 +32,10 @@ to access the file content, it's possible to decode it in the deploy script, usi
 echo {{file}} | base64 -d > /ccp_data/input
 ```
 
-then simply read the file  `/ccp_data/input` during the method execution 
+then simply read the file  `/ccp_data/input` during the method execution
 
 ```json
 {
-    ...
-
     "additionalParameters": {
         "parameters": [
             {
@@ -58,6 +55,6 @@ then simply read the file  `/ccp_data/input` during the method execution
 }
 ```
 
-to work with bigger files, refer to the Input file workspace section
+to work with bigger files, refer to the [Input file workspace section](./03_3_input_file_workspace_public.md)
 
-example: [Gate Cloud Hate And Abuse Detection ](../ccp_methods/Gate%20Cloud%20Hate%20And%20Abuse%20Detection-1.0.0.json)
+example: [Python example](../methods_examples/Python%20example-1.0.0.json)

source/methods_development/03_2_input_urls.md

@@ -0,0 +1,62 @@
+# Input URL
+
+The input with format "url" allows the user to provide a direct link to a file or resource to be processed during the method execution.
+
+![input url](../_static/imgs/methods_development/input_url.png)
+
+
+
+```json
+{
+    "inputs": {
+        "file_url": {
+            "id": "file_url",
+            "title": "Input URL",
+            "description": "The URL of the file to be processed",
+            "minOccurs": 1,
+            "maxOccurs": 1,
+            "schema": {
+                "type": "string",
+                "format": "url",
+                "contentMediaType": "text/plain",
+                "default": ""
+            }
+        }
+    }
+}
+
+The input URL must point to a publicly accessible resource, as the method execution will fetch the content from the URL during runtime.
+
+To process the file content, the provided URL has to be retrieved in the deploy script using a command like:
+
+`wget {{file_url}} -O /ccp_data/input`
+
+or 
+
+`curl -L {{file_url}} -o /ccp_data/input`
+
+Once the file is retrieved, the method can read it from the /ccp_data/input path during execution.
+
+```json
+{
+    "additionalParameters": {
+        "parameters": [
+            {
+                "name": "deploy-script",
+                "value": [
+                    "wget {{file_url}} -O /ccp_data/input"
+                ]
+            },
+            {
+                "name": "execute-script",
+                "value": [
+                    "wget {{baseurl}}/{{service}} --post-file /ccp_data/input  -O /ccp_data/annotated.json"
+                ]
+            }
+        ]
+    }
+}
+```
+
+Example: [WorldCloud Urls](../methods_examples/WordCloud%20Urls-1.0.0.json)
+

source/methods_development/03_3_input_file_workspace_public.md

@@ -1,12 +1,12 @@
-# download file from workspace
+# download public file from workspace
 
 Authenticate the service on Keycloak, retrieve a token
 then download the file using the authentication token
 
-![](./imgs/workspace_input_file.png)
+![](../_static/imgs/methods_development/input_file.png)
 
 The input form allows to select from the user workspace files
-![](./imgs/workspace_select_file.png)
+![](../_static/imgs/methods_development/workspace_select_file.png)
 
 method json
 
@@ -60,4 +60,4 @@ curl $1 -o /ccp_data/inputFile.csv -H "Authorization: Bearer $TOKEN"
 echo "Downloaded"
 ```
 
-example: [Ariadne Dutch Archaeology Named Entity Recognizer](../ccp_methods/Ariadne%20Dutch%20Archaeology%20Named%20Entity%20Recognizer-1.0.0.json)
+example: [Ariadne Dutch Archaeology Named Entity Recognizer](../methods_ccp/Ariadne%20Dutch%20Archaeology%20Named%20Entity%20Recognizer-1.0.0.json)

source/methods_development/03_4_credentials.md

@@ -1,9 +1,9 @@
-# Credentials
+# Credentials and Remote authentication
 
 * Input of credentials in a secret input form
 * Definition of an execute-script that uses the provided credentials to authenticate on an external service
 
-![credentials](./imgs/credentials.png)
+![credentials](../_static/imgs/methods_development/input_credentials.png)
 
 ```json
 {
@@ -35,4 +35,4 @@
 }
 ```
 
-example: [Gate Cloud Hate And Abuse Detection ](../ccp_methods/Gate%20Cloud%20Hate%20And%20Abuse%20Detection-1.0.0.json)
+example: [Bash Example](../methods_examples/Bash%20Example-1.0.0.json)

source/methods_development/03_5_input_file_workspace_private.md

@@ -0,0 +1,63 @@
+# download private file from workspace
+
+Authenticate the service on Keycloak, retrieve a token
+then download the file using the authentication token
+
+![](../_static/imgs/methods_development/input_file.png)
+
+The input form allows to select from the user workspace files
+![](../_static/imgs/methods_development/workspace_select_file.png)
+
+method json
+
+```json
+{
+    "inputs": {
+        "inputFile": {
+            "id": "inputFile",
+            "title": "inputFile",
+            "description": "Input CSV file()",
+            "minOccurs": 1,
+            "maxOccurs": 1,
+            "schema": {
+                "type": "string",
+                "format": "remotefile",
+                "default": "",
+                "contentMediaType": "text/csv"
+            }
+        }
+    },
+    "additionalParameters": {
+        "parameters": [
+            {
+                "name": "deploy-script",
+                "value": [
+                    "./download.sh {{inputFile}}"
+                ]
+            },
+            {
+                "name": "execute-script",
+                "value": [
+                    "python XXX.py /ccp_data/inputFile.csv [[ARGS]]",
+                    "cp -f result.csv /ccp_data/"
+                ]
+            }
+        ]
+    }
+}
+```
+
+download implementation:
+the docker image contains the download script [download.sh](../support_files/download.sh]`
+
+```sh 
+echo "Getting token"
+TOKEN=$(curl -X POST $ccpiamurl -d grant_type=refresh_token -d client_id=$ccpclientid -d refresh_token=$ccprefreshtoken -H "X-D4Science-Context: $ccpcontext" | jq -r '."access_token"')
+
+echo Downloading /ccp_data/inputFile.csv
+curl $1 -o /ccp_data/inputFile.csv -H "Authorization: Bearer $TOKEN"
+
+echo "Downloaded"
+```
+
+example: [Ariadne Dutch Archaeology Named Entity Recognizer](../methods_ccp/Ariadne%20Dutch%20Archaeology%20Named%20Entity%20Recognizer-1.0.0.json)

source/methods_development/03_6_input_checklist.md

@@ -3,7 +3,7 @@
 * definition of a checklist with given string set
 * use of the checked params in the execute-script
 
-![checklist](./imgs/checklist.png)
+![checklist](../_static/imgs/methods_development/input_checklist.png )
 
 ```json
 {
@@ -74,4 +74,4 @@
 }
 ```
 
-example: [Gate Cloud Hate And Abuse Detection ](../ccp_methods/Gate%20Cloud%20Hate%20And%20Abuse%20Detection-1.0.0.json)
+example: [Gate Cloud Hate And Abuse Detection ](../methods_ccp/Gate%20Cloud%20Hate%20And%20Abuse%20Detection-1.0.0.json)

source/methods_development/index.rst

@@ -0,0 +1,8 @@
+Methods development reference
+=============================
+
+.. toctree::
+   :glob:
+   :includehidden:
+
+   *

source/methods_examples/Python example-1.0.0.json

@@ -65,17 +65,6 @@
             {
                 "name": "undeploy-script",
                 "value": []
-            },
-            {
-                "name": "lifecycle",
-                "value": [
-                    {
-                        "time": "2024-10-04T15:30:06.183Z",
-                        "type": "created",
-                        "user": "https://accounts.d4science.org/auth/admin/realms/d4science/users/b20300b9-d2d7-4ef8-b164-917f661f7ee0",
-                        "context": "%2Fd4science.research-infrastructures.eu%2FD4Research%2FFOSSR-Lab"
-                    }
-                ]
             }
         ]
     },

source/methods_examples/WordCloud Urls-1.0.0.json

@@ -0,0 +1,122 @@
+{
+    "title": "WordCloud Urls",
+    "description": "word cloud generator. rif https://github.com/amueller/word_cloud",
+    "version": "1.0.0",
+    "jobControlOptions": "async-execute",
+    "metadata": [
+        {
+            "title": "Alfredo Oliviero",
+            "role": "author",
+            "href": "https://accounts.dev.d4science.org/auth/admin/realms/d4science/users/b9969b51-578f-4b69-a53f-2c8adf9efcc7"
+        },
+        {
+            "role": "category",
+            "title": "WordCloud"
+        },
+        {
+            "role": "category",
+            "title": "Examples"
+        }
+    ],
+    "inputs": {
+        "ccpimage": {
+            "id": "ccpimage",
+            "title": "Runtime",
+            "description": "The image of the runtime to use for method execution. This depends on the infrastructure specific protocol for interacting with registries.",
+            "minOccurs": 1,
+            "maxOccurs": 1,
+            "schema": {
+                "type": "string",
+                "format": "url",
+                "contentMediaType": "text/plain",
+                "default": "python:3.9.19",
+                "readOnly": false
+            }
+        },
+        "file_url": {
+            "id": "file_url",
+            "title": "file",
+            "description": "the file to analyze",
+            "minOccurs": 1,
+            "maxOccurs": 1,
+            "schema": {
+                "type": "string",
+                "format": "url",
+                "contentMediaType": "text/plain",
+                "default": "https://data.d4science.net/WWxr"
+            }
+        }
+    },
+    "outputs": {
+        "stdout": {
+            "id": "stdout",
+            "title": "Standard output",
+            "description": "Standard output channel",
+            "minOccurs": 1,
+            "maxOccurs": 1,
+            "metadata": [
+                {
+                    "role": "file",
+                    "title": "stdout",
+                    "href": "stdout"
+                }
+            ],
+            "schema": {
+                "type": "string",
+                "contentMediaType": "text/plain"
+            }
+        },
+        "stderr": {
+            "id": "stderr",
+            "title": "Standard error",
+            "description": "Standard error channel",
+            "minOccurs": 1,
+            "maxOccurs": 1,
+            "metadata": [
+                {
+                    "role": "file",
+                    "title": "stderr",
+                    "href": "stderr"
+                }
+            ],
+            "schema": {
+                "type": "string",
+                "contentMediaType": "text/plain"
+            }
+        }
+    },
+    "additionalParameters": {
+        "parameters": [
+            {
+                "name": "deploy-script",
+                "value": [
+                    "pip install --root-user-action=ignore --disable-pip-version-check wordcloud",
+                    "curl -L {{file_url}} -o /ccp_data/input.txt",
+                    ""
+                ]
+            },
+            {
+                "name": "execute-script",
+                "value": [
+                    "wordcloud_cli --text /ccp_data/input.txt --imagefile /ccp_data/wordcloud.png"
+                ]
+            },
+            {
+                "name": "undeploy-script",
+                "value": []
+            }
+        ]
+    },
+    "links": [
+        {
+            "rel": "compatibleWith",
+            "title": "D4Science development Infrastructure",
+            "href": "infrastructures/d4science-dev-swarm"
+        }
+    ],
+    "keywords": [
+        "python",
+        "wordcloud"
+    ],
+    "id": "e21acefd-d584-454d-bb53-a0e5b8b77e1a"
+}

source/methods_examples/WordCloud-1.0.0.json

@@ -0,0 +1,113 @@
+{
+    "title": "WordCloud",
+    "description": "word cloud generator. rif https://github.com/amueller/word_cloud",
+    "version": "1.0.0",
+    "jobControlOptions": "async-execute",
+    "metadata": [
+        {
+            "title": "Alfredo Oliviero",
+            "role": "author",
+            "href": "https://accounts.d4science.org/auth/admin/realms/d4science/users/b20300b9-d2d7-4ef8-b164-917f661f7ee0"
+        }
+    ],
+    "inputs": {
+        "ccpimage": {
+            "id": "ccpimage",
+            "title": "Runtime",
+            "description": "The image of the runtime to use for method execution. This depends on the infrastructure specific protocol for interacting with registries.",
+            "minOccurs": 1,
+            "maxOccurs": 1,
+            "schema": {
+                "type": "string",
+                "format": "url",
+                "contentMediaType": "text/plain",
+                "default": "python:3.9.19",
+                "readOnly": false
+            }
+        },
+        "file": {
+            "id": "file",
+            "title": "file",
+            "description": "the file to analyze",
+            "minOccurs": 1,
+            "maxOccurs": 1,
+            "schema": {
+                "type": "string",
+                "format": "file",
+                "contentMediaType": "text/plain",
+                "default": ""
+            }
+        }
+    },
+    "outputs": {
+        "stdout": {
+            "id": "stdout",
+            "title": "Standard output",
+            "description": "Standard output channel",
+            "minOccurs": 1,
+            "maxOccurs": 1,
+            "metadata": [
+                {
+                    "role": "file",
+                    "title": "stdout",
+                    "href": "stdout"
+                }
+            ],
+            "schema": {
+                "type": "string",
+                "contentMediaType": "text/plain"
+            }
+        },
+        "stderr": {
+            "id": "stderr",
+            "title": "Standard error",
+            "description": "Standard error channel",
+            "minOccurs": 1,
+            "maxOccurs": 1,
+            "metadata": [
+                {
+                    "role": "file",
+                    "title": "stderr",
+                    "href": "stderr"
+                }
+            ],
+            "schema": {
+                "type": "string",
+                "contentMediaType": "text/plain"
+            }
+        }
+    },
+    "additionalParameters": {
+        "parameters": [
+            {
+                "name": "deploy-script",
+                "value": [
+                    "echo {{file}} | base64 -d > /ccp_data/input.txt",
+                    "pip install --root-user-action=ignore wordcloud"
+                ]
+            },
+            {
+                "name": "execute-script",
+                "value": [
+                    "wordcloud_cli --text /ccp_data/input.txt --imagefile /ccp_data/wordcloud.png"
+                ]
+            },
+            {
+                "name": "undeploy-script",
+                "value": []
+            }
+        ]
+    },
+    "links": [
+        {
+            "rel": "compatibleWith",
+            "title": "D4Science production Infrastructure",
+            "href": "infrastructures/d4science-prod-swarm"
+        }
+    ],
+    "keywords": [
+        "python",
+        "wordcloud"
+    ],
+    "id": "2d415c9e-3aff-4b9b-b278-3a27d7f212df"
+}

source/quickstart/01.0_analytics_engine.md

@@ -6,11 +6,11 @@ The *Analytics Engine* section consists of two main pages:
 
 1. The **[Cloud Computing Platform (CCP)](01.2_engine_ccp.md)**: This page allows users to execute predefined methods and monitor their execution.
 
-   ![Cloud Computing Platform](./imgs/analytics_engine_ccp.png)
+   ![Cloud Computing Platform](../_static/imgs/quickstart/analytics_engine_ccp.png)
 
-2. The **[Method/Algorithm Importer](01.3_engine_edit.md)**: This page is used to define new methods, as well as manage and edit existing ones.
+2. The **[Method/Algorithm Importer](01.5_engine_edit.md)**: This page is used to define new methods, as well as manage and edit existing ones.
 
-   ![Method/Algorithm Importer](./imgs/analytics_engine_importer.png)
+   ![Method/Algorithm Importer](../_static/imgs/quickstart/analytics_engine_importer.png)
 
 ---
 

source/quickstart/01.1_methods_list.md

@@ -2,7 +2,7 @@
 
 Both the *Cloud Computing Platform (CCP)* and the *Analytics Engine* display the **Methods List** section in the left column, which contains all methods available in the current environment. These methods are grouped by category for easy navigation.
 
-![Methods List](./imgs/methods_list.png)
+![Methods List](../_static/imgs/quickstart/methods_list.png)
 
 ---
 
@@ -16,7 +16,7 @@ Each method is described by the following attributes:
 - **Tags**: Relevant tags that classify the method.
 - **Infrastructures**: Indicates the infrastructures where the method can be executed (usually, *D4Science Production Infrastructure*).
 
-![Method Detail](./imgs/method_shared.png)
+![Method Detail](../_static/imgs/quickstart/method_shared.png)
 
 ---
 
@@ -24,7 +24,7 @@ Each method is described by the following attributes:
 
 To the right of each method, a series of buttons allows you to execute, edit, or manage the method.
 
-![Method Buttons](./imgs/method_buttons.png)
+![Method Buttons](../_static/imgs/quickstart/method_buttons.png)
 
 The available actions for each method depend on several factors, including:
 
@@ -38,32 +38,32 @@ Some buttons may be hidden based on these conditions. Below are the possible act
 
 ### Action Buttons
 
-#### ![Play button](./imgs/method_button_play.png)  **Play Button**
+#### ![Play button](../_static/imgs/quickstart/method_button_play.png)  **Play Button**
 
 Loads the method into the execution form, where users can configure the input and start the execution.
 
 > **Availability**: Shown only on the *Cloud Computing Platform (CCP)* page. The method must have at least one available infrastructure in the current VRE (Virtual Research Environment) to be executed; otherwise, the button will be hidden.
 
-#### ![Edit button](./imgs/method_button_edit.png)  **Edit Button**
+#### ![Edit button](../_static/imgs/quickstart/method_button_edit.png)  **Edit Button**
 
 Loads the method into the editing form, where users can manage and modify the method’s definition.
 
 > **Availability**: Shown only on the *Method/Algorithm Importer* page.
 
-#### ![Download button](./imgs/method_button_download.png)  **Download Button**
+#### ![Download button](../_static/imgs/quickstart/method_button_download.png)  **Download Button**
 
 Downloads the JSON file that defines the method.
 
-#### ![Archive button](./imgs/method_button_archive.png) **Archive Button**
+#### ![Archive button](../_static/imgs/quickstart/method_button_archive.png) **Archive Button**
 
 Similar to the download button, but saves the method’s JSON file to the user’s workspace instead of downloading it to the local machine
 
-#### ![Share button](./imgs/method_button_share.png) **Share Button**
+#### ![Share button](../_static/imgs/quickstart/method_button_share.png) **Share Button**
 
 Clicking this button shares the method with others in the environment.
 
 > **Availability**: Appears only for methods defined by the user that have not yet been shared with the community. Once shared, the method becomes visible to users across the relevant VREs, depending on the permissions set by the original creator.
 
-#### ![Archived label](./imgs/method_button_shared.png) **Archived Label**
+#### ![Archived label](../_static/imgs/quickstart/method_button_shared.png) **Archived Label**
 
 Indicates that the method has been shared with the community.

source/quickstart/01.2_engine_ccp.md

@@ -8,100 +8,32 @@ The page is composed of:
 - **Method Execution form** section, used to configure and start the execution of a selected method.
 - **Execution Monitor** section, which tracks scheduled, pending, and finished executions.
 
-![Cloud Computing Platform (CCP)](./imgs/analytics_engine_ccp.png)
+![Cloud Computing Platform (CCP)](../_static/imgs/quickstart/analytics_engine_ccp.png)
 
 ---
 
-## **Methods List**
+## Methods List section
 
 When the **Cloud Computing Platform (CCP)** page is first opened, the execution form will be empty. To select a method for execution, use the **Methods List** section on the left. Methods can be selected by:
 
-- Clicking the **Play** button ( ![play button](./imgs/method_button_play.png) ) next to a method to load it into the execution form.
+- Clicking the **Play** button ( ![play button](../_static/imgs/quickstart/method_button_play.png) ) next to a method to load it into the execution form.
 - Dragging the method from the list and dropping it into the execution form.
 
+--- 
 
----
-
-## **Method Execution form** Section
+## Method Execution section
 
 Once a method is selected, the **Method Execution form** is populated with the inputs and options specific to the selected method. 
 
-Each method has its own predefined inputs, runtime, and parameters, which are defined during the method's configuration.
+The widget will shot method's predefined inputs, runtime, and parameters defined during its configuration.
 
-The *runtime* environment is defined by the `ccpimage` input, which specifies the Docker image to be used. This runtime is **usually immutable**, meaning users typically cannot modify it during execution, but in certain cases, flexibility may be allowed depending on the method's configuration.
+The user can populate the methods's parameters and start the execution, as it will explained in section [Method execution](./01.3_method_execution.md)
 
-### **Input Parameters**
+--- 
 
-The input parameters required for execution vary based on the method's definition. Common types of inputs include:
-
-- **`ccpimage` (Runtime)**: Specifies the Docker image or environment in which the method will run. This is predefined via the `ccpimage` input and is *usually immutable*.
-- **Input Files/URLs**: Paths to input files or URLs, such as CSV files or other required data formats, depending on the method's needs.
-- **Input Parameters**: These include specific configuration values such as probabilities, number of iterations, or other settings required by the algorithm.
-- **Annotations**: Metadata or tags associated with the execution, which can also appear in the output results, providing traceability across executions.
-
-### **Options**
-
-In the **Options** section, users can select the automatic archiving behavior for their execution results. The following options are available:
-
-- **Automatically archive whole execution provenance**: This option will add to the archive the entire execution context, including inputs, parameters, and results.
-- **Automatically archive uncompressed outputs only**: This option will archive only the final outputs in an uncompressed format, without including additional details.
-- **Do not archive anything automatically**: No automatic archiving of the execution results will be done.
-
-### **Outputs**
-
-You can configure what type of logs to capture during the method execution:
-
-- **Standard output**: Enable this option to capture all normal output messages generated by the method.
-- **Standard error**: Enable this option to capture any error or warning messages encountered during execution.
-
-
-![options](./imgs/cpp_execution_options.png)
-
----
-
-## **Executing a Method**
-
-Once the method is selected and configured, you can execute it by clicking the **Execute** button. Here's how the execution process works:
-
-1. **Scheduling of execution**: After clicking *Execute*, CCP will process the method and parameters and scheduel an execution
-
-2. **Start of Execution**:  begin by pulling the necessary Docker image (as specified by `ccpimage`), installing any dependencies, and running the method.
-
-3. **Monitoring Progress**: The execution can be monitored in real time via the **Execution Monitor** (described below), where status updates and logs will be displayed.
-
-4. **Outputs**: When the method completes, a link to download the output files, such as processed data, logs, or visualizations, will be provided. If provenance tracking is enabled, all execution details will be saved automatically.
-
----
-## **Execution Monitor** Section
+## Execution Monitor section
 
 The **Execution Monitor** provides a detailed view of method executions, tracking their status from the moment they are initiated until they complete. It is composed of two main sections: **Live Executions** and **Archived Executions**.
 
-![Execution Monitor](./imgs/ccp_execution_monitor.png)
+![Execution Monitor](../_static/imgs/quickstart/ccp_execution_monitor.png)
 
-### **Live Executions**
-
-This tab displays the current status of running or recently finished executions. Each running execution shows:
-
-- **Method Name and Version**: Identifies the method being executed.
-- **Status**: The execution's current phase, which may include:
-  - Accepted: The method has been accepted and is queued for execution.
-  - Running: The method is currently being executed.
-  - Successful: The execution has completed successfully.
-  - Failed: An error occurred during execution.
-- **Timestamp**: Indicates the time the execution was accepted and the time of the last update.
-- **Infrastructure**: The infrastructure used to run the method (e.g., D4Science Production Infrastructure).
-- **Runtime**: The Docker image used (e.g., python:3.9.19).
-  
-Additional features in this tab include:
-
-- **Output Files**: When the execution completes, a link (`outputs/output.zip`) will appear, allowing users to download the execution results.
-- **Generate Code for...**: Generates a file in the selected programming language (e.g., Python, R, Julia) that authenticates the user and programmatically starts the execution.
-- **Direct Link**: Provides a direct link that opens the CCP page directly to the specific execution instance.
-- **Logs**: Real-time logs showing progress and any warnings or errors.
-- **Cancel Execution**: Allows you to stop an ongoing execution if needed.
-- **Re-submit**: Restarts a previously executed method with the same configuration.
-- **Delete button**: To delete the current execution from the list of executions
-
-### **Archived Executions**
-
-From the **Live Executions panel**, you can archive completed executions. Once archived, the execution will be moved to the **Archived Executions panel**, where you can review the history of past executions.

source/quickstart/01.3_method_execution.md

@@ -0,0 +1,38 @@
+
+# Method Execution
+
+Once a method is selected, the **Method Execution form** is populated with the inputs and options specific to the selected method.
+
+Each method has its own predefined inputs, runtime, and parameters, which are defined during the method's configuration.
+
+## Runtime
+
+The *runtime* environment is defined by the `ccpimage` input, which specifies the Docker image to be used. This runtime is **usually immutable**, meaning users typically cannot modify it during execution, but in certain cases, flexibility may be allowed depending on the method's configuration.
+
+## Input Parameters
+
+The input parameters required for execution vary based on the method's definition. Common types of inputs include:
+
+- **`ccpimage` (Runtime)**: Specifies the Docker image or environment in which the method will run. This is predefined via the `ccpimage` input and is *usually immutable*.
+- **Input Files/URLs**: Paths to input files or URLs, such as CSV files or other required data formats, depending on the method's needs.
+- **Input Parameters**: These include specific configuration values such as probabilities, number of iterations, or other settings required by the algorithm.
+- **Annotations**: Metadata or tags associated with the execution, which can also appear in the output results, providing traceability across executions.
+
+## Options
+
+In the **Options** section, users can select the automatic archiving behavior for their execution results. The following options are available:
+
+- **Automatically archive whole execution provenance**: This option will add to the archive the entire execution context, including inputs, parameters, and results.
+- **Automatically archive uncompressed outputs only**: This option will archive only the final outputs in an uncompressed format, without including additional details.
+- **Do not archive anything automatically**: No automatic archiving of the execution results will be done.
+
+## Outputs
+
+You can configure what type of logs to capture during the method execution:
+
+- **Standard output**: Enable this option to capture all normal output messages generated by the method.
+- **Standard error**: Enable this option to capture any error or warning messages encountered during execution.
+
+
+![options](../_static/imgs/quickstart/cpp_execution_options.png)
+

source/quickstart/01.4_execution_monitor.md

@@ -0,0 +1,33 @@
+# Execution Monitor Section
+
+The **Execution Monitor** provides a detailed view of method executions, tracking their status from the moment they are initiated until they complete. It is composed of two main sections: **Live Executions** and **Archived Executions**.
+
+![Execution Monitor](../_static/imgs/quickstart/ccp_execution_monitor.png)
+
+## Live Executions
+
+This tab displays the current status of running or recently finished executions. Each running execution shows:
+
+- **Method Name and Version**: Identifies the method being executed.
+- **Status**: The execution's current phase, which may include:
+  - Accepted: The method has been accepted and is queued for execution.
+  - Running: The method is currently being executed.
+  - Successful: The execution has completed successfully.
+  - Failed: An error occurred during execution.
+- **Timestamp**: Indicates the time the execution was accepted and the time of the last update.
+- **Infrastructure**: The infrastructure used to run the method (e.g., D4Science Production Infrastructure).
+- **Runtime**: The Docker image used (e.g., python:3.9.19).
+  
+Additional features in this tab include:
+
+- **Output Files**: When the execution completes, a link (`outputs/output.zip`) will appear, allowing users to download the execution results.
+- **Generate Code for...**: Generates a file in the selected programming language (e.g., Python, R, Julia) that authenticates the user and programmatically starts the execution.
+- **Direct Link**: Provides a direct link that opens the CCP page directly to the specific execution instance.
+- **Logs**: Real-time logs showing progress and any warnings or errors.
+- **Cancel Execution**: Allows you to stop an ongoing execution if needed.
+- **Re-submit**: Restarts a previously executed method with the same configuration.
+- **Delete button**: To delete the current execution from the list of executions
+
+## Archived Executions
+
+From the **Live Executions panel**, you can archive completed executions. Once archived, the execution will be moved to the **Archived Executions panel**, where you can review the history of past executions.

source/quickstart/01.5_engine_edit.md

@@ -1,4 +1,4 @@
-# Method/Algorithm Importer
+# Method/Algorithm Importer page
 
 The **Method/Algorithm Importer** page allows users to create new computational methods or edit existing ones. When the page is opened, the form will be empty, providing the opportunity to define a new method. Alternatively, users can select an existing method from the **Methods List** to modify it.
 
@@ -7,27 +7,27 @@ The page is composed of several sections:
 - **Methods List**: Displays all available methods categorized for easy navigation.
 - **Method Editor**: The section where users can create or edit methods, define inputs, outputs, and specify execution scripts.
 
-![Method Importer Overview](./imgs/analytics_engine_importer.png)
+![Method Importer Overview](../_static/imgs/quickstart/analytics_engine_importer.png)
 
 
 ---
 
 
 
-## **Methods List**
+## Methods List
 
-When the **Method/Algorithm Importer (CCP)** page is opened, the edit form will be empty by default, allowing users to create a new method. To edit an existing method, use the **Methods List** section on the left. Methods can be selected by:
+When the **Method/Algorithm Importer (CCP)** page is opened, the edit form will be empty by default, allowing users to create a new method. To edit an existing method, use the [**Methods List**](./01.1_methods_list.md) section on the left. Methods can be selected by:
 
-- Clicking the **Edit** button ( ![edit button](./imgs/method_button_edit.png) ) next to a method to load it into the execution form.
+- Clicking the **Edit** button ( ![edit button](../_static/imgs/quickstart/method_button_edit.png) ) next to a method to load it into the execution form.
 - Dragging the method from the list and dropping it into the execution form.
 
 ---
 
-## **Method Editor**
+## Method Editor
 
 The **Method Editor** is where users can create new methods or edit existing ones. It consists of the following components:
 
-### **Basic Information**
+### Basic Information
 - **Title**: The name of the method.
 - **Version**: The version number of the method.
 - **Description**: A brief description of the method's purpose.
@@ -35,7 +35,7 @@ The **Method Editor** is where users can create new methods or edit existing one
 - **Categories**: The broader category the method belongs to.
 - **Compatible Infrastructures**: Lists the infrastructures where the method can be executed.
 
-## **Saving and Managing Methods**
+## Saving and Managing Methods
 
 Once the method is defined, users can:
 
@@ -46,7 +46,7 @@ Once the method is defined, users can:
 Saved methods will appear in the **Methods List** .
 ---
 
-### **Inputs**
+### Inputs
 
 Users can define the inputs required by the method. Each input is configurable, including:
 
@@ -57,11 +57,11 @@ Users can define the inputs required by the method. Each input is configurable,
 
 Each input can be added or removed using the **Add** (+) and **Remove** (Trash) buttons.
 
-### **Defining a New Input**
+### Defining a New Input
 
-By pressing the **Plus** button ![Plus Button](./imgs/edit_button_plus.png) in the **Inputs** section, the interface for defining a new input field is created.
+By pressing the **Plus** button ![Plus Button](../_static/imgs/quickstart/edit_button_plus.png) in the **Inputs** section, the interface for defining a new input field is created.
 
-![New Input Form](./imgs/edit_new_input.png)
+![New Input Form](../_static/imgs/quickstart/edit_new_input.png)
 
 The following fields can be configured for the new input:
 
@@ -76,7 +76,7 @@ The following fields can be configured for the new input:
 - **Default Value**: The value that will be used as default if no input is provided during execution.
 
 
-#### **String Inputs**
+#### String Inputs
 
 selecting *String* as type , The available options fo **Format** for enumerated types include:
 
@@ -93,7 +93,7 @@ selecting *String* as type , The available options fo **Format** for enumerated
 - **Mime Type**: Defines the expected media type for the input, such as `text/plain` or `application/json`.
 
 
-#### **Enumeration Inputs**
+#### Enumeration Inputs
 
 When **Enumerated** is selected as the type, users must define a comma-separated list of options. The execution interface will then display these options as a list or a set of checkboxes for selection.
 
@@ -101,15 +101,15 @@ When **Enumerated** is selected as the type, users must define a comma-separated
   - **single_choice**: Allows the user to select one option from the list.
   - **multi_choice**: Allows the user to select multiple options from the list.
 
-![Input enumerated](./imgs/edit_input_enumerated.png)
+![Input enumerated](../_static/imgs/quickstart/edit_input_enumerated.png)
 
 ---
 
-### **Outputs**
+### Outputs
 
 The **Outputs** section allows users to define the outputs that will be generated by the method. By default, this section is empty, but users can add standard or custom outputs by using the buttons available.
 
-![Outputs Section](./imgs/edit_output.png)
+![Outputs Section](../_static/imgs/quickstart/edit_output.png)
 
 - **Green Button**: Adds `stdout` (standard output) to capture all standard messages generated by the method during execution.
 - **Red Button**: Adds `stderr` (standard error) to capture warnings and error messages generated during execution.
@@ -119,7 +119,7 @@ Each output can be removed by clicking the **Trash** icon next to the correspond
 
 ---
 
-### **Scripts**
+### Scripts
 
 
 The **Scripts** section allows users to define the commands necessary for deploying and executing the method. The inputs provided in the form must be pre-processed based on their format before being used in these scripts.
@@ -135,15 +135,15 @@ In the upcoming sections, we will discuss how to handle and process inputs in bo
 
 By using placeholders, the scripts dynamically incorporate the values provided during the execution configuration, allowing for flexible and reusable methods.
 
-![Method Editor](./imgs/edit_script.png)
+![Method Editor](../_static/imgs/quickstart/edit_script.png)
 
 ---
 
-### **Outputs**
+### Outputs
 
 The **Outputs** section allows users to define the outputs that will be generated by the method. By default, this section is empty, but users can add standard or custom outputs by using the buttons available.
 
-![Outputs Section](./imgs/output_section.png)
+![Outputs Section](../_static/imgs/quickstart/edit_output.png)
 
 - **Green Button**: Adds `stdout` (standard output) to capture all standard messages generated by the method during execution.
 - **Red Button**: Adds `stderr` (standard error) to capture warnings and error messages generated during execution.
@@ -153,7 +153,7 @@ Each output can be removed by clicking the **Trash** icon next to the correspond
 
 
 
-## **Deploy and Execute Scripts**
+## Deploy and Execute Scripts
 
 The **Deploy and Execute Scripts** allow users to configure method-specific scripts for deployment and execution.
 
@@ -162,6 +162,4 @@ The **Deploy and Execute Scripts** allow users to configure method-specific scri
 
 These scripts ensure that the method is correctly initialized and executed on the selected infrastructure.
 
-![Scripts Section](./imgs/scripts_section.png)
-
----
+![Scripts Section](../_static/imgs/quickstart/edit_script.png)

source/quickstart/35_input_file_workspace.md

@@ -1,6 +0,0 @@
-# Input file workspace
-
-Gate Cloud Annie Named Entity Recognizer 1.0.0 Marco Lettere
-
-[Gate Cloud Annie Named Entity Recognizer](../ccp_methods/Gate%20Cloud%20Annie%20Named%20Entity%20Recognizer-1.0.0.json)
-

source/quickstart/46_remote_api_auth_credentials.md

@@ -1,47 +0,0 @@
-# Client for External Api, with Basic Auth Credentials:
-
-
-![input credentials ](./imgs/credentials.png)
-```json
-{
-    "inputs": {
-        "credentials": {
-            "id": "credentials",
-            "title": "Credentials",
-            "description": "The Basic auth credentials",
-            "minOccurs": 1,
-            "maxOccurs": 1,
-            "schema": {
-                "type": "string",
-                "format": "secret",
-                "contentMediaType": "text/plain",
-                "default": "..."
-            }
-        }
-    },
-     "additionalParameters": {
-    "parameters": [
-      {
-        "name": "deploy-script",
-        "value": [
-          "echo {{file}} | base64 -d > /ccp_data/input"
-        ]
-      },
-      {
-        "name": "execute-script",
-        "value": [
-          "wget {{baseurl}}/{{service}}?annotations={{annotations}} --post-file /ccp_data/input --header \"Authorization: Basic {{credentials}}\" --header \"Content-Type: {{contenttype}}\" --header \"Accept: application/json\" -O /ccp_data/annotated.json"
-        ]
-      },
-      {
-        "name": "undeploy-script",
-        "value": []
-      }
-    ]
-  }
-
-}
-```
-
-example [Gate Cloud Hate And Abuse Detection ](../ccp_methods/Gate%20Cloud%20Hate%20And%20Abuse%20Detection-1.0.0.json)
-

source/quickstart/index.rst

@@ -1,5 +1,5 @@
 Quickstart
-=========
+==========
 
 .. toctree::
    :glob:

source/usermanual/04_methods.md

@@ -265,66 +265,3 @@ The following two environmental valuables provide context for the execution.
 * **ccptaskid** is the index of the replica (1-based) when multiple replicas are requested with the input parameter ccpreplicas. This can be used to customise the behavior of a replica like accessing a slice of a dataset or writing output to different files.
 
 The following variables are related to the authentication and authorization context of the Method execution. They can be used to access D4Science services in a secure and convenient way also for very long lasting executions.  
-
-
-## How To 
-
-### How to use `ccptaskid` to separate output of different replicas to different files in `execute-script`.
-
-``` bash
-    mkdir -p /ccp_data/output && echo $RANDOM >> /ccp_data/output/`printenv ccptaskid`.txt
-``` 
-
-* **ccpiamurl** is the URL of the Identity management service.
-* **ccpclientid** is the client_id to be used for requesting token renewal.
-* **ccprefreshtoken** is a refresh token by which new access tokens can be requested.
-* **ccpcontext** represents the context (VO or VRE) in which the Method execution has been requested.
-
-As an example the following Python code shows how to use the variables to request a token renewal.
-
-*How to request a login token and an UMA token for accessing D4Science service from inside Method code*
-```python
-#Get auth info from env variables
-refreshtoken = os.environ["ccprefreshtoken"]
-context = os.environ["ccpcontext"]
-clientid = os.environ["ccpclientid"]
-iam = os.environ["ccpiamurl"]
-
-#Auth related functions
-logindata = { 'grant_type' : 'refresh_token', 'client_id' : clientid, 'refresh_token' : refreshtoken}
-loginheaders = { "Accept" : "application/json", "Content-Type" : "application/x-www-form-urlencoded"}
-
-umadata = { 'grant_type' : 'urn:ietf:params:oauth:grant-type:uma-ticket', 'audience' : context}
-umaheaders = { "Accept" : "application/json", "Content-Type" : "application/x-www-form-urlencoded"}
-
-def getToken():
-    # login with offline_token
-    resp1 = requests.post(iam, data=logindata, headers=loginheaders)
-    jwt = resp1.json()
-    #get UMA token for context
-    umaheaders["Authorization"] = "Bearer " + jwt["access_token"]
-    resp2 = requests.post(iam, data=umadata, headers=umaheaders)
-    return resp2.json()["access_token"]
-
-# Get valid token for context
-tok = getToken()
-
-# List VRE fodler content
-vrefolder = requests.get(workspace + "/vrefolder", headers={"Accept" : "application/json", "Authorization" : "Bearer " + tok}).json()s
-```
-
-A similar example in bash could have the following form:
-
-### How to download a file from the D4Science workspace from a private URL passed as input parameter
-
-```bash    
-echo "Getting token"
-TOKEN=$(curl -X POST $ccpiamurl -d grant_type=refresh_token -d client_id=$ccpclientid -d refresh_token=$ccprefreshtoken -H "X-D4Science-Context: $ccpcontext" | jq -r '."access_token"')
-
-echo Downloading $wslink to $inputfile
-curl -L $wslink -o $inputfile -H "Authorization: Bearer $TOKEN"
-
-echo "Downloaded"
-```
-
-A special folder is provided in the Runtime to a Method execution for storing output files. Files are currently the only way for a Method to output results by value. The folder is named **/ccp_data** and all the files written to this folder are returned in the context of the Execution as a zip archive.

source/usermanual/05_howtos.md

@@ -0,0 +1,63 @@
+
+
+# How To 
+
+## How to use `ccptaskid` to separate output of different replicas to different files in `execute-script`.
+
+``` bash
+    mkdir -p /ccp_data/output && echo $RANDOM >> /ccp_data/output/`printenv ccptaskid`.txt
+``` 
+
+* **ccpiamurl** is the URL of the Identity management service.
+* **ccpclientid** is the client_id to be used for requesting token renewal.
+* **ccprefreshtoken** is a refresh token by which new access tokens can be requested.
+* **ccpcontext** represents the context (VO or VRE) in which the Method execution has been requested.
+
+As an example the following Python code shows how to use the variables to request a token renewal.
+
+*How to request a login token and an UMA token for accessing D4Science service from inside Method code*
+```python
+#Get auth info from env variables
+refreshtoken = os.environ["ccprefreshtoken"]
+context = os.environ["ccpcontext"]
+clientid = os.environ["ccpclientid"]
+iam = os.environ["ccpiamurl"]
+
+#Auth related functions
+logindata = { 'grant_type' : 'refresh_token', 'client_id' : clientid, 'refresh_token' : refreshtoken}
+loginheaders = { "Accept" : "application/json", "Content-Type" : "application/x-www-form-urlencoded"}
+
+umadata = { 'grant_type' : 'urn:ietf:params:oauth:grant-type:uma-ticket', 'audience' : context}
+umaheaders = { "Accept" : "application/json", "Content-Type" : "application/x-www-form-urlencoded"}
+
+def getToken():
+    # login with offline_token
+    resp1 = requests.post(iam, data=logindata, headers=loginheaders)
+    jwt = resp1.json()
+    #get UMA token for context
+    umaheaders["Authorization"] = "Bearer " + jwt["access_token"]
+    resp2 = requests.post(iam, data=umadata, headers=umaheaders)
+    return resp2.json()["access_token"]
+
+# Get valid token for context
+tok = getToken()
+
+# List VRE fodler content
+vrefolder = requests.get(workspace + "/vrefolder", headers={"Accept" : "application/json", "Authorization" : "Bearer " + tok}).json()s
+```
+
+A similar example in bash could have the following form:
+
+### How to download a file from the D4Science workspace from a private URL passed as input parameter
+
+```bash    
+echo "Getting token"
+TOKEN=$(curl -X POST $ccpiamurl -d grant_type=refresh_token -d client_id=$ccpclientid -d refresh_token=$ccprefreshtoken -H "X-D4Science-Context: $ccpcontext" | jq -r '."access_token"')
+
+echo Downloading $wslink to $inputfile
+curl -L $wslink -o $inputfile -H "Authorization: Bearer $TOKEN"
+
+echo "Downloaded"
+```
+
+A special folder is provided in the Runtime to a Method execution for storing output files. Files are currently the only way for a Method to output results by value. The folder is named **/ccp_data** and all the files written to this folder are returned in the context of the Execution as a zip archive.