D4Science CKAN 2.10 Docker Image

Docker-based deployment of CKAN 2.10.5 for the D4Science catalogue infrastructure

This deployment is based on the mjanez/ckan-docker project, adapted and extended for D4Science infrastructure requirements.

Table of Contents

• Prerequisites
• Installation
• Running the Services
• Service Endpoints
• Testing the Installation
• Harvesting External Catalogs
• Configuration
• Architecture
• Troubleshooting
• References

Prerequisites

• Docker >= 20.10
• Docker Compose >= 2.0
• Git
• At least 4GB RAM and 10GB disk space available

Installation

1. Clone the repository:

   git clone gitea@code-repo.d4science.org:D4ScienceCatalogue/docker-ckan.git
   cd docker-ckan
   

2. Create your environment configuration:

   cp .env.example .env
   

3. (Optional) Edit .env to customize your deployment settings (database credentials, ports, etc.)

Running/Stop the Services

Development Environment

Linux:

docker compose -f docker-compose.dev.yml up -d --build

sudo docker compose -f docker-compose.dev.yml down

macOS (Intel/Apple Silicon):

DOCKER_DEFAULT_PLATFORM=linux/amd64 docker compose -f docker-compose.dev.yml up -d --build

DOCKER_DEFAULT_PLATFORM=linux/amd64 docker compose -f docker-compose.dev.yml down

Production Environment

Linux:

docker compose up -d --build

macOS:

DOCKER_DEFAULT_PLATFORM=linux/amd64 docker compose up -d --build

Monitoring Logs

To monitor the services in real-time:

docker compose logs -f ckan

To view all service logs:

docker compose logs -f

Service Endpoints

By default, the following services are exposed:

Service	URL	Description
CKAN (via nginx)	http://localhost:81/catalog	Main CKAN web interface
CKAN (direct)	http://localhost:5000	Direct CKAN application (dev only)
NGINX (HTTP)	http://localhost:81	Reverse proxy
NGINX (HTTPS)	https://localhost:8443	Secure reverse proxy
Solr	http://localhost:8983	Search engine admin interface

Note: Ports can be customized in the .env file.

Testing the Installation

1. Access CKAN

Navigate to: http://localhost:81/catalog

2. Default Credentials

For testing purposes, use:

• Username: ckan_admin
• Password: test1234

⚠️ Important: Change these credentials in production environments!

3. Test Other Services

• Solr Admin UI: http://localhost:8983/solr

Harvesting External Catalogs

To harvest metadata from external CKAN instances:

1. Create an Organization

   • Navigate to: http://localhost:81/catalog/organization/
   • Go to: Organizations → Add Organization
   • Fill in the organization details and save

2. Configure a Harvest Source

   • Navigate to: http://localhost:81/catalog/harvest/new
   • Fill in all required fields:
     • URL: The external CKAN instance URL (e.g., https://ckan-sobigdata.d4science.org/)
     • Source Type: Select CKAN
     • Frequency: Select Manually for testing
     • Organization: Select the organization created in step 1
   • Save the harvest source

3. Run the Harvest

   • Navigate to the harvest source admin page
   • Click "Admin" and then "Reharvest" to manually trigger the harvest

4. Monitor Harvest Jobs

   • Check the harvest job status and logs in the admin interface
   • Review harvested datasets in your organization

Configuration

Environment Variables

Key configuration options in .env:

# CKAN Configuration
CKAN_SITE_URL=http://localhost:81/catalog
CKAN_SYSADMIN_NAME=ckan_admin
CKAN_SYSADMIN_PASSWORD=test1234
CKAN_SYSADMIN_EMAIL=admin@example.com

# Database Configuration
POSTGRES_PASSWORD=ckan
CKAN_DB_USER=ckan_default
CKAN_DB_PASSWORD=ckan

# Ports
NGINX_PORT_HOST=81
NGINX_SSLPORT_HOST=8443

# Theme Branch (D4Science customization)
THEME_BRANCH=main

Custom Extensions

This deployment includes the following CKAN extensions:

• ckanext-harvest - Metadata harvesting framework
• ckanext-spatial - Spatial metadata and search
• ckanext-dcat - DCAT/DCAT-AP metadata support
• ckanext-xloader - Fast data loading
• ckanext-scheming - Custom metadata schemas
• ckanext-pages - Static pages management
• ckanext-hierarchy - Organization hierarchies
• ckanext-d4science_theme - D4Science custom theme
• And more (see Dockerfile for the complete list)

Architecture

graph TD
  Browser[Browser] -->|HTTP/HTTPS| NGINX[NGINX<br/>Reverse Proxy]
  NGINX --> CKAN[CKAN]
  CKAN --> Solr[Solr]
  CKAN --> Redis[Redis]
  CKAN --> PostgreSQL[PostgreSQL]
  CKAN --> Volumes[Volumes]

Troubleshooting

Container Won't Start

Check logs:

docker compose logs ckan

Verify all services are running:

docker compose ps

Permission Issues

If you encounter permission errors related to i18n files:

docker exec -it <ckan_container_name> chown -R ckan:ckan /srv/app/src/ckan/ckan/i18n/

Database Connection Issues

Ensure PostgreSQL is fully initialized:

docker compose logs db

Reset the database (⚠️ destroys all data):

docker compose down -v
docker compose up -d

Port Conflicts

If ports are already in use, modify them in .env:

NGINX_PORT_HOST=8081
NGINX_SSLPORT_HOST=8444

Rebuild Images

To rebuild images from scratch:

docker compose down
docker compose build --no-cache
docker compose up -d

Authors

• Francesco Mangiacrapa (ORCID) Computer Scientist at ISTI-CNR Infrascience Group

• Marco Procaccini Computer Scientist at IGG-CNR

References

• Original Docker Setup: https://github.com/mjanez/ckan-docker
• CKAN Documentation: https://docs.ckan.org/
• CKAN Extensions: https://extensions.ckan.org/
• D4Science Infrastructure: https://www.d4science.org/

License

This project is licensed under the terms specified in the LICENSE.md file.

Support

For issues related to D4Science infrastructure, contact the D4Science catalogue team.