ccp.docs/source/methods_development/02.2_basic_python.md

Python algoritms

Python methods can be implemented compiling the ccpimage field with a standard python docker image (es python:3.9.19)

We can then define an input repository field

and configure the deploy script to git clone the repository and install the dependencies, using the placeholder {{repository}} to refer the input repository field

git clone {{repository}}
cd sortapp
pip install -r requirements.txt

then the run script will run it passing the required parameters

{
    "inputs": {
        "ccpimage": {
            "id": "ccpimage",
            "title": "Runtime",
            "description": "the standard 3.19.19 python docker image",
            "minOccurs": 1,
            "maxOccurs": 1,
            "schema": {
                "type": "string",
                "format": "url",
                "contentMediaType": "text/plain",
                "default": "python:3.9.19",
                "readOnly": true
            }
        }
    }
}

Execution Parameters

all the parameters and configurations needed for the execution of the algorithms are defined in the input sections

{
    "inputs": {
        "baseurl": {
            "id": "baseurl",
            "title": "Base URL",
            "description": "The base URL of the REST API",
            "minOccurs": 1,
            "maxOccurs": 1,
            "schema": {
                "type": "string",
                "format": "url",
                "contentMediaType": "text/plain",
                "default": "https://REST_API_URL/process/",
                "readOnly": true
            }
        },
        "service": {
            "id": "service",
            "title": "Service name",
            "description": "The name of the service",
            "minOccurs": 1,
            "maxOccurs": 1,
            "schema": {
                "type": "string",
                "format": null,
                "contentMediaType": "text/plain",
                "default": "my_custom_service"
            }
        },
        "file": {
            "id": "file",
            "title": "Input file",
            "description": "The input file to be annotated",
            "minOccurs": 1,
            "maxOccurs": 1,
            "schema": {
                "type": "string",
                "format": "file",
                "contentMediaType": "text/plain",
                "default": ""
            }
        },
        "contenttype": {
            "id": "contenttype",
            "title": "Content type",
            "description": "The content type of the input file",
            "minOccurs": 1,
            "maxOccurs": 1,
            "schema": {
                "type": "string",
                "format": null,
                "contentMediaType": "text/plain",
                "default": "text/plain",
                "enum": [
                    "text/plain",
                    "application/pdf",
                    "text/html"
                ]
            }
        },
        "credentials": {
            "id": "credentials",
            "title": "Credentials",
            "description": "The Basic auth credentials",
            "minOccurs": 1,
            "maxOccurs": 1,
            "schema": {
                "type": "string",
                "format": "secret",
                "contentMediaType": "text/plain",
                "default": "XXXYYYZZZ"
            }
        }
    }
}

the "readOnly": true/false parameter defines what is binded in the definiton of the ccp method (readonly=true, only the baseurl input in the given example) and what is parametric (readonly=false, or not defined in the input schema)

the given example uses special input parameters that will be explained in next sections

• credentials
• file upload

Execution

the implementation of the algorithms is totally defined in the deploy-script and in the execute-script

{
    "additionalParameters": {
        "parameters": [
            {
                "name": "deploy-script",
                "value": [
                    "echo {{file}} | base64 -d > /ccp_data/input"
                ]
            },
            {
                "name": "execute-script",
                "value": [
                    "wget {{baseurl}}/{{service}}?annotations={{annotations}} --post-file /ccp_data/input --header \"Authorization: Basic {{credentials}}\" --header \"Content-Type: {{contenttype}}\" --header \"Accept: application/json\" -O /ccp_data/annotated.json"
                ]
            },
            {
                "name": "undeploy-script",
                "value": []
            }
        ]
    }
}

full example

{
    "title": "Python example",
    "description": "basic python example",
    "version": "1.0.0",
    "jobControlOptions": "async-execute",
    "keywords": [
        "python",
        "example"
    ],
    "id": "2a59f2ec-9771-44d0-80dc-13f3deb863ed",
    "metadata": [
        {
            "role": "category",
            "title": "examples"
        },
        {
            "title": "Alfredo Oliviero",
            "role": "author",
            "href": "https://accounts.d4science.org/auth/admin/realms/d4science/users/b20300b9-d2d7-4ef8-b164-917f661f7ee0"
        }
    ],
    "inputs": {
        "ccpimage": {
            "id": "ccpimage",
            "title": "Runtime",
            "description": "The image of the runtime to use for method execution. This depends on the infrastructure specific protocol for interacting with registries.",
            "minOccurs": 1,
            "maxOccurs": 1,
            "schema": {
                "type": "string",
                "format": "none",
                "contentMediaType": "text/plain",
                "default": "python:3.9.19",
                "readOnly": true
            }
        },
        "repository": {
            "id": "repository",
            "title": "repository url",
            "description": "url of the repository of the python application",
            "minOccurs": 1,
            "maxOccurs": 1,
            "schema": {
                "type": "string",
                "format": "url",
                "contentMediaType": "text/plain",
                "default": "https://code-repo.d4science.org/gCubeSystem/sortapp.git",
                "readOnly": true
            }
        }
    },
    "outputs": {},
    "additionalParameters": {
        "parameters": [
            {
                "name": "deploy-script",
                "value": [
                    "git clone {{repository}}",
                    "cd sortapp",
                    "pip install -r requirements.txt"
                ]
            },
            {
                "name": "execute-script",
                "value": [
                    "cd sortapp/",
                    "python sortapp.py"
                ]
            },
            {
                "name": "undeploy-script",
                "value": []
            },
            {
                "name": "lifecycle",
                "value": [
                    {
                        "time": "2024-10-04T15:30:06.183Z",
                        "type": "created",
                        "user": "https://accounts.d4science.org/auth/admin/realms/d4science/users/b20300b9-d2d7-4ef8-b164-917f661f7ee0",
                        "context": "%2Fd4science.research-infrastructures.eu%2FD4Research%2FFOSSR-Lab"
                    }
                ]
            }
        ]
    },
    "links": [
        {
            "rel": "compatibleWith",
            "title": "D4Science production Infrastructure",
            "href": "infrastructures/d4science-prod-swarm"
        }
    ]
}

example: WordCloud