engine ccp

This commit is contained in:
Alfredo Oliviero 2024-10-09 17:03:14 +02:00
parent c41cd7b228
commit ee77bbeaa7
3 changed files with 52 additions and 30 deletions

View File

@ -1,10 +1,10 @@
# Cloud Computing Platform (CCP) Page
The **Cloud Computing Platform (CCP)** page allows users to execute predefined methods and monitor their execution within a cloud-based infrastructure.
The **Cloud Computing Platform (CCP)** page allows users to execute the methods defined for the current environment and monitor their execution within the cloud-based infrastructure.
The page is composed of:
- **Methods List** section on the left, displaying all available methods (see [Methods List](./07_methods_list.md)).
- **Methods List** section on the left, displaying all available methods (see [Methods List](./01.1_methods_list.md)).
- **Method Execution form** section, used to configure and start the execution of a selected method.
- **Execution Monitor** section, which tracks scheduled, pending, and finished executions.
@ -19,29 +19,43 @@ When the **Cloud Computing Platform (CCP)** page is first opened, the execution
- Clicking the **Play** button ( ![play button](./imgs/method_button_play.png) ) next to a method to load it into the execution form.
- Dragging the method from the list and dropping it into the execution form.
Each method has its own predefined inputs, runtime, and parameters, which are configured during the method's creation. The *runtime* environment is defined by the `ccpimage` input, which specifies the Docker image to be used. This runtime is **usually immutable**, meaning users typically cannot modify it during execution, but in certain cases, flexibility may be allowed depending on the method's configuration.
---
## **Method Execution form** Section
Once a method is selected, the **Method Execution form** is populated with the inputs and options specific to the selected method. These parameters and inputs depend entirely on how the method was defined when it was integrated into CCP.
Once a method is selected, the **Method Execution form** is populated with the inputs and options specific to the selected method.
Each method has its own predefined inputs, runtime, and parameters, which are defined during the method's configuration.
The *runtime* environment is defined by the `ccpimage` input, which specifies the Docker image to be used. This runtime is **usually immutable**, meaning users typically cannot modify it during execution, but in certain cases, flexibility may be allowed depending on the method's configuration.
### **Input Parameters**
The input parameters required for execution vary based on the method's definition. Common types of inputs include:
- **Runtime (`ccpimage`)**: Specifies the Docker image or environment in which the method will run. This is predefined via the `ccpimage` input and is **usually immutable**.
- **`ccpimage` (Runtime)**: Specifies the Docker image or environment in which the method will run. This is predefined via the `ccpimage` input and is *usually immutable*.
- **Input Files/URLs**: Paths to input files or URLs, such as CSV files or other required data formats, depending on the method's needs.
- **Parameters**: These include specific configuration values such as probabilities, number of iterations, or other settings required by the algorithm.
- **Input Parameters**: These include specific configuration values such as probabilities, number of iterations, or other settings required by the algorithm.
- **Annotations**: Metadata or tags associated with the execution, which can also appear in the output results, providing traceability across executions.
### **Options**
Depending on the method, additional options may be available:
In the **Options** section, users can select the automatic archiving behavior for their execution results. The following options are available:
- **Provenance Tracking**: Users can opt to automatically archive the execution details, including the provenance of the inputs, parameters, and results. This helps in keeping track of the execution history for reproducibility.
- **Error Handling**: Users can choose to report and log any errors or warnings encountered during the execution.
- **Automatically archive whole execution provenance**: This option will add to the archive the entire execution context, including inputs, parameters, and results.
- **Automatically archive uncompressed outputs only**: This option will archive only the final outputs in an uncompressed format, without including additional details.
- **Do not archive anything automatically**: No automatic archiving of the execution results will be done.
### **Outputs**
You can configure what type of logs to capture during the method execution:
- **Standard output**: Enable this option to capture all normal output messages generated by the method.
- **Standard error**: Enable this option to capture any error or warning messages encountered during execution.
![options](./imgs/cpp_execution_options.png)
---
@ -49,37 +63,45 @@ Depending on the method, additional options may be available:
Once the method is selected and configured, you can execute it by clicking the **Execute** button. Here's how the execution process works:
1. **Start of Execution**: After clicking *Execute*, CCP will begin by pulling the necessary Docker image (as specified by `ccpimage`), installing any dependencies, and running the method.
2. **Monitoring Progress**: The execution can be monitored in real time via the **Execution Monitor** (described below), where status updates and logs will be displayed.
3. **Outputs**: When the method completes, a link to download the output files, such as processed data, logs, or visualizations, will be provided. If provenance tracking is enabled, all execution details will be saved automatically.
1. **Scheduling of execution**: After clicking *Execute*, CCP will process the method and parameters and scheduel an execution
2. **Start of Execution**: begin by pulling the necessary Docker image (as specified by `ccpimage`), installing any dependencies, and running the method.
3. **Monitoring Progress**: The execution can be monitored in real time via the **Execution Monitor** (described below), where status updates and logs will be displayed.
4. **Outputs**: When the method completes, a link to download the output files, such as processed data, logs, or visualizations, will be provided. If provenance tracking is enabled, all execution details will be saved automatically.
---
## **Execution Monitor** Section
The **Execution Monitor** tracks the status of both active and past method executions. It provides two primary views:
The **Execution Monitor** provides a detailed view of method executions, tracking their status from the moment they are initiated until they complete. It is composed of two main sections: **Live Executions** and **Archived Executions**.
![Execution Monitor](./imgs/ccp_execution_monitor.png)
### **Live Executions**
This tab displays the current status of running or recently finished executions. Key details include:
This tab displays the current status of running or recently finished executions. Each running execution shows:
- **Method Name and Version**: Identifies the method being executed.
- **Status**: Indicates the execution's current phase (e.g., pending, running, successful, failed).
- **Timestamp**: Shows when the execution started and when the last update was received.
- **Annotations**: Metadata and tags associated with the execution, which can be used to trace and organize multiple executions.
- **Status**: The execution's current phase, which may include:
- Accepted: The method has been accepted and is queued for execution.
- Running: The method is currently being executed.
- Successful: The execution has completed successfully.
- Failed: An error occurred during execution.
- **Timestamp**: Indicates the time the execution was accepted and the time of the last update.
- **Infrastructure**: The infrastructure used to run the method (e.g., D4Science Production Infrastructure).
- **Runtime**: The Docker image used (e.g., python:3.9.19).
Additional features in this tab include:
Actions available in this tab include:
- **Cancel Execution**: Stop a running execution.
- **View Logs**: Access live logs to monitor the installation of dependencies, execution progress, warnings, and errors.
- **Resubmit**: Restart a previously executed method with the same configuration.
- **Output Files**: When the execution completes, a link (`outputs/output.zip`) will appear, allowing users to download the execution results.
- **Generate Code for...**: Generates a file in the selected programming language (e.g., Python, R, Julia) that authenticates the user and programmatically starts the execution.
- **Direct Link**: Provides a direct link that opens the CCP page directly to the specific execution instance.
- **Logs**: Real-time logs showing progress and any warnings or errors.
- **Cancel Execution**: Allows you to stop an ongoing execution if needed.
- **Re-submit**: Restarts a previously executed method with the same configuration.
- **Delete button**: To delete the current execution from the list of executions
### **Archived Executions**
This tab contains the history of all completed executions, whether successful or failed. Each execution includes:
- **Execution Name**: The name of the method.
- **Status**: The final result of the execution.
- **Details**: Includes inputs, outputs, annotations, and logs.
Users can download the outputs for further analysis or re-execute the method with modified parameters. This history is useful for troubleshooting,
From the **Live Executions panel**, you can archive completed executions. Once archived, the execution will be moved to the **Archived Executions panel**, where you can review the history of past executions.

Binary file not shown.

After

Width:  |  Height:  |  Size: 223 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 109 KiB