D4Science Python Library

A Python library that provides authentication, configuration, and client functionalities for the D4Science infrastructure. This project offers a modern, Typer-based Command Line Interface (CLI) for interacting with D4Science services, alongside a robust library for programmatic access.

Key Features

• Modern CLI: A powerful and user-friendly CLI built with Typer, featuring:
  • Modular, discoverable commands (storagehub, core, auth).
  • Rich, colorized help and output.
  • Global options for configuration, context, and verbosity.
  • JSON output for easy scripting and integration.
• Centralized Configuration: Manage multiple environments (e.g., dev, prod) and contexts seamlessly.
• OAuth2 Authentication: Securely obtain and manage access tokens.
• Service Clients: High-level clients for interacting with D4Science services like the StorageHub.

Project Structure

• d4science/main.py: The main entry point for the Typer-based CLI.
• d4science/core/: Core functionalities, including configuration management (config.py) and the core CLI module (cli_typer.py).
• d4science/auth/: Authentication logic (auth.py) and the auth CLI module (cli_typer.py).
• d4science/storagehub/: Client for the StorageHub service (client.py) and its comprehensive CLI modules (cli/).
• .d4science/: Directory for project-specific configuration and credentials files.
• configs/: Directory for example configuration files.
• ~/.d4science/: User-specific configuration and credentials directory.

Built With

• Python 3.x
• Typer - For the Command Line Interface
• Pydantic - For data validation and settings management
• Requests - HTTP library
• Requests-OAuthlib - OAuth management

Getting Started

Installation

1. Clone the repository:

   git clone <repository-url>
   cd d4science-python-library
   

2. Install the library. Choose one of the following methods:

   A) Editable Install (for development)

   This method links the installed package back to your source code. Any changes you make to the code will be immediately available without reinstalling. This is the recommended method for developers.

   pip install -e .
   

   B) Standalone Install

   This method builds and installs the package in your Python environment, just like installing a package from PyPI. This is suitable for users who want to use the library without modifying its source code.

   pip install .
   

   Option B: Global installation (for system-wide CLI access):

   On macOS with Homebrew-managed Python:

   brew install pipx
   pipx install -e .
   

   On other systems:

   pip install --user -e .
   

   If you encounter "externally-managed-environment" error, use:

   pip install --user -e . --break-system-packages
   

Configuration

The library uses a simple and consistent configuration structure:

Configuration Files:

• Config files: config.<ENV>.json (e.g., config.dev.json, config.prod.json)
• Credentials files: auth.<ENV>.json (e.g., auth.dev.json, auth.prod.json)

Configuration Locations (in priority order):

1. CLI-specified file: --config-filename /path/to/config.json
2. Environment variable: D4S_CONFIG_PATH=/path/to/config.json
3. Project directory: ./.d4science/config.<ENV>.json
4. Project fallback: ./.d4science.json
5. User directory: ~/.d4science/config.<ENV>.json

Setup Example:

1. Create project configuration directory:

   mkdir .d4science
   

2. Copy and edit the example configuration:

   cp configs/config.example.json .d4science/config.dev.json
   

3. Edit .d4science/config.dev.json to set your desired context and parameters.

CLI Usage

The CLI is now available via the d4science command.

General Help:

d4science --help

Core Commands (Configuration):

# Create a new configuration file for the 'dev' environment in project .d4science/
d4science --env dev core config create --base-path https://api.d4science.org

# Create configuration in user home ~/.d4science/ directory instead of project
d4science --env dev core config create --base-path https://api.d4science.org --user-home

# Create configuration in a specific custom path
d4science --env dev core config create --base-path https://api.d4science.org --output /path/to/my-config.json

# Create configuration in a specific directory (filename will be auto-generated)
d4science --env dev core config create --base-path https://api.d4science.org --output /path/to/directory/

# Create and overwrite existing configuration (if any)
d4science --env dev core config create --base-path https://api.d4science.org --force

# Show help for the create command
d4science --env dev core config create --help

# Show the currently loaded configuration
d4science --env dev core config show

Authentication Commands (Login & Credentials):

# Log in to get an access token
d4science --env dev auth login

# Get just the access token as string
d4science --env dev auth get-token

# Save your credentials for the 'dev' environment (stored in .d4science/auth.dev.json)
d4science --env dev auth credentials save

# Show saved credentials for current environment
d4science --env dev auth credentials show

# List all available credentials (searches in project .d4science/ and user ~/.d4science/)
d4science auth credentials list

# List credentials only in user home ~/.d4science/ directory
d4science auth credentials list --user-home

# List credentials only in current project .d4science/ directory
d4science auth credentials list --current

# List credentials in a specific custom path
d4science auth credentials list --path /path/to/configs

# Filter credentials by specific environment
d4science auth credentials list --filter-env dev

# List credentials without showing usernames
d4science auth credentials list --no-show-usernames

# Combined options: filter by env and search only in home
d4science auth credentials list --filter-env prod --user-home

# Clear saved credentials for current environment
d4science --env dev auth credentials clear

StorageHub Commands:

# List items in the root of your workspace
d4science --env dev storagehub items ls /

# Create a new folder
d4science --env dev storagehub items mkdir /new-folder

# Upload a file
d4science --env dev storagehub items upload /path/to/local/file.txt /new-folder/

Documentation

See additional documentation in the docs/ folder or specific instructions.md files within each module.

Change log

See CHANGELOG.md.

Authors

• Alfredo Oliviero (ORCID) - ISTI-CNR Infrascience Group - ISTI-CNR D4Science Group
• Biagio Peccerillo (ORCID) - ISTI-CNR Infrascience Group - ISTI-CNR D4Science Group

How to Cite this Software

If you use this software in a publication, please cite:

@Manual{,
  title  = {D4Science Python Library},
  author = {{Infrascience Group}},
  organization = {ISTI - CNR},
  address = {Pisa, Italy},
  year   = 2025,
  note   = {Accessed from repository},
  url    = {https://www.gcube-system.org/}
}

License

This project is licensed under the license EUPL V. 1.2 - see the LICENSE.md file for details.

About the gCube Framework

This software is part of the gCubeFramework: an open-source software toolkit used for building and operating Hybrid Data Infrastructures enabling the dynamic deployment of Virtual Research Environments by favouring the realisation of reuse oriented policies.

The projects leading to this software have received funding from a series of European Union programmes see FUNDING.md