Compare commits

...

No commits in common. "main" and "main" have entirely different histories.
main ... main

652 changed files with 314150 additions and 21132 deletions

317
.env.example Normal file
View File

@ -0,0 +1,317 @@
# Base
APP_DIR=/srv/app
# Sets Docker Compose project name to avoid resource conflicts between projects. Defaults to the folder name "ckan-docker" if not set.
#COMPOSE_PROJECT_NAME=ckan-docker-mytheme
# Dev settings
USE_HTTPS_FOR_DEV=false
CKAN_DEV_COMPOSE_SERVICE=ckan-dev
## ckan-pycsw unverified mode (true/false). SSL certificate from host will download if SSL_UNVERIFIED_MODE=true, to avoid SSL error when certificate was self-signed.
SSL_UNVERIFIED_MODE=false
# Host Ports
CKAN_PORT_HOST=5000
PYCSW_PORT_HOST=8000
# Common proxy server for Apache or NGINX.
# Change all the PROXY_SERVER_URL_PORT_HOST to PROXY_SERVER_HTTPS_PORT_HOST if you can use HTTPS instead of HTTP (Only for NGINX and not development compose)
PROXY_SERVER_HTTP_PORT_HOST=81
PROXY_SERVER_HTTPS_PORT_HOST=8443
# Ports for NGINX (matches HTTP/HTTPS ports)
NGINX_PORT_HOST=${PROXY_SERVER_HTTP_PORT_HOST}
NGINX_SSLPORT_HOST=${PROXY_SERVER_HTTPS_PORT_HOST}
# Port for Apache
APACHE_PORT_HOST=${PROXY_SERVER_HTTP_PORT_HOST}
# Server port for the proxy server and CKAN urls. Better using the SSL port.
# (If needed for Development Mode) uncomment the following line:
# PROXY_SERVER_URL_PORT_HOST=${PROXY_SERVER_HTTP_PORT_HOST}
# (If needed for Development Mode) And comment the following line:
PROXY_SERVER_URL_PORT_HOST=${PROXY_SERVER_HTTPS_PORT_HOST}
# URL protocol. Change to http if you dont use SSL
PROXY_SERVER_PROTOCOL=https
# Solr
SOLR_IMAGE_VERSION=2.9-solr9-spatial
SOLR_PORT=8983
SOLR_CKAN_DATABASE=ckan
CKAN_SOLR_URL=http://solr:${SOLR_PORT}/solr/${SOLR_CKAN_DATABASE}
TEST_CKAN_SOLR_URL=http://solr:${SOLR_PORT}/solr/${SOLR_CKAN_DATABASE}
# Redis
REDIS_VERSION=7-alpine
REDIS_PORT=6379
REDIS_CKAN_DATABASE=1
CKAN_REDIS_URL=redis://redis:${REDIS_PORT}/${REDIS_CKAN_DATABASE}
TEST_CKAN_REDIS_URL=redis://redis:${REDIS_PORT}/${REDIS_CKAN_DATABASE}
# Google Analytics
# ANALYTICS_ID="UA-1010101-1"
# NGINX
NGINX_PORT=80
NGINX_SSLPORT=443
NGINX_LOG_DIR=/var/log/nginx
# Apache HTTP Server
APACHE_VERSION=2.4-alpine
APACHE_PORT=80
APACHE_LOG_DIR=/var/log/apache
#APACHE_LOG_LEVEL=debug
#NGINX/APACHE
## Check CKAN__ROOT_PATH and CKANEXT__DCAT__BASE_URI and CKANEXT__SCHEMINGDCAT_GEOMETADATA_BASE_URI. If you don't need to use domain locations, it is better to use the nginx configuration. Leave blank or use the root `/`. If you dont need PROXY_SERVER_URL_PORT_HOST (e.g. is 80), edit the PROXY_SERVER_URL and remove :${PROXY_SERVER_URL_PORT_HOST}
PROXY_SERVER_NAME=localhost
PROXY_SERVER_URL=${PROXY_SERVER_PROTOCOL}://${PROXY_SERVER_NAME}:${PROXY_SERVER_URL_PORT_HOST}
PROXY_CKAN_LOCATION=/catalog
PROXY_PYCSW_LOCATION=/csw
#CKAN_STATIC_PATH=/catalog/static
# pycsw
PYCSW_PORT=8000
CKAN_URL=${PROXY_SERVER_URL}${PROXY_CKAN_LOCATION}
PYCSW_URL=${PROXY_SERVER_URL}${PROXY_PYCSW_LOCATION}
# SCHEMAS: ckan2pycsw/model/dataset.py - Dataset type
PYCSW_CKAN_SCHEMA=iso19139_geodcatap
PYCSW_OUPUT_SCHEMA=iso19139_inspire
# ckan-pycsw schedule
## ckan2pycsw days between each scheduler job
PYCSW_CRON_DAYS_INTERVAL=2
# ckan2pycsw hour of start of the scheduler job (0-23)
PYCSW_CRON_HOUR_START=4
## Timezone
TZ=UTC
# CKAN databases
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=postgres
POSTGRES_HOST=db
POSTGRES_LOG_LEVEL=DEBUG1
CKAN_DB_USER=ckandbuser
CKAN_DB_PASSWORD=ckandbpassword
CKAN_DB=ckandb
DATASTORE_READONLY_USER=datastore_ro
DATASTORE_READONLY_PASSWORD=datastore
DATASTORE_DB=datastore
CKAN_SQLALCHEMY_URL=postgresql://${CKAN_DB_USER}:${CKAN_DB_PASSWORD}@${POSTGRES_HOST}/${CKAN_DB}
CKAN_DATASTORE_WRITE_URL=postgresql://${CKAN_DB_USER}:${CKAN_DB_PASSWORD}@${POSTGRES_HOST}/${DATASTORE_DB}
CKAN_DATASTORE_READ_URL=postgresql://${DATASTORE_READONLY_USER}:${DATASTORE_READONLY_PASSWORD}@${POSTGRES_HOST}/${DATASTORE_DB}
# Test database connections
CKAN_TEST_DB=ckan_test
DATASTORE_TEST_DB=datastore_test
TEST_CKAN_SQLALCHEMY_URL=postgres://${CKAN_DB_USER}:${CKAN_DB_PASSWORD}@${POSTGRES_HOST}/${CKAN_TEST_DB}
TEST_CKAN_DATASTORE_WRITE_URL=postgresql://${CKAN_DB_USER}:${CKAN_DB_PASSWORD}@${POSTGRES_HOST}/${DATASTORE_TEST_DB}
TEST_CKAN_DATASTORE_READ_URL=postgresql://${DATASTORE_READONLY_USER}:${DATASTORE_READONLY_PASSWORD}@${POSTGRES_HOST}/${DATASTORE_TEST_DB}
# CKAN core
## If use docker-compose.ghcr.yml only "*.*.*" versions available in: https://github.com/mjanez/ckan-docker/pkgs/container/ckan-docker
CKAN_VERSION=2.10.5
CKAN_SITE_ID=default
# CKAN_SITE_URL = http:/ or https:/ + PROXY_SERVER_NAME. Optionally the APACHE_HOST_PORT if different from 80
CKAN_SITE_URL=${PROXY_SERVER_URL}
#CKAN__ROOT_PATH=${PROXY_CKAN_LOCATION}/{{LANG}}
CKAN__ROOT_PATH=${PROXY_CKAN_LOCATION}/{{LANG}}
CKAN__FAVICON=${PROXY_CKAN_LOCATION}/base/images/ckan.ico
CKAN__SITE_LOGO=${PROXY_CKAN_LOCATION}/images/default/ckan-logo.png
# CKAN site info
CKAN__SITE_TITLE="CKAN Open Data"
CKAN__SITE_DESCRIPTION="ckan-docker *spatial Open Data portal"
# Using [#en#], [#es#], etc. to define the language of the text as needed.
CKAN__SITE_INTRO_TEXT="[#en#][![ckan-docker-spatial](https://img.shields.io/badge/ckan_docker_spatial-CKAN_${CKAN_VERSION}-brightgreen?style=social&logo=github#center)](https://github.com/mjanez/ckan-docker)\n[`ckan-docker *spatial`](https://github.com/mjanez/ckan-docker) is a ready-to-use development environment for [CKAN](https://github.com/ckan/ckan), the world’s leading open-source data portal platform.[#es#][![ckan-docker-spatial](https://img.shields.io/badge/ckan_docker_spatial-CKAN_${CKAN_VERSION}-brightgreen?style=social&logo=github#center)](https://github.com/mjanez/ckan-docker)\n[`ckan-docker *spatial`](https://github.com/mjanez/ckan-docker) es un entorno de desarrollo listo para usar para [CKAN](https://github.com/ckan/ckan), la plataforma de portales de datos de código abierto líder en el mundo."
CKAN__SITE_ABOUT=${CKAN__SITE_INTRO_TEXT}
# Custom licenses that are aligned with DCAT-AP (Also works with DEV Mode)
CKAN___LICENSES_GROUP_URL=file://${APP_DIR}/src/ckanext-d4science-theme/licenses/gcube-data-catalogue-licenses.json
CKAN___BEAKER__SESSION__SECRET=CHANGE_ME
# See https://docs.ckan.org/en/latest/maintaining/configuration.html#api-token-settings
CKAN___API_TOKEN__JWT__ENCODE__SECRET=string:CHANGE_ME
CKAN___API_TOKEN__JWT__DECODE__SECRET=string:CHANGE_ME
CKAN_SYSADMIN_NAME=ckan_admin
CKAN_SYSADMIN_PASSWORD=test1234
CKAN_SYSADMIN_EMAIL=your_emai@${PROXY_SERVER_NAME}
CKAN_STORAGE_PATH=/var/lib/ckan
CKAN_LOGS_PATH=/var/log
CKAN_LOG_LEVEL=DEBUG
# SMTP settings
CKAN__SMTP_ENABLED=false
CKAN_SMTP_SERVER=smtp.corporateict.domain:25
CKAN_SMTP_STARTTLS=true
CKAN_SMTP_USER=user
CKAN_SMTP_PASSWORD=pass
CKAN_SMTP_MAIL_FROM=ckan@${PROXY_SERVER_NAME}
## Customize which text formats the text_view plugin will show
CKAN__PREVIEW__JSON_FORMATS="json jsonld"
# html htm rdf+xml owl+xml xml n3 n-triples turtle plain atom csv tsv rss txt json
CKAN__PREVIEW__XML_FORMATS="xml rdf rdf+xml owl+xml atom rss turtle ttl n3 n-triples"
CKAN__PREVIEW__TEXT_FORMATS="text plain text/plain text/turtle rss txt"
CKAN__PREVIEW__LOADABLE="html htm rdf+xml owl+xml xml n3 n-triples turtle plain atom csv tsv rss txt json arcgis_rest"
# ckanext-spatial: Allow Solr local params: https://github.com/ckan/ckanext-spatial/issues/328
CKAN__SEARCH__SOLR_ALLOWED_QUERY_PARSERS=field
# CORS Settings. If true, all origins will be allowed (the response header Access-Control-Allow-Origin is set to ‘*’). If false, only origins from the ckan.cors.origin_whitelist setting will be allowed.
CKAN__CORS__ORIGIN_ALLOW_ALL=false
CKAN__CORS__ORIGIN_WHITELIST=""
# Default number of facets shown in search results. Default 10.
SEARCH__FACETS__DEFAULT=10000
# Enable or disable the DataStore SQL search backend. Default is False.
CKAN__DATASTORE__SQLSEARCH__ENABLED=true
## Resource Proxy settings
### Preview size limit, default: 1MB
CKAN__RESOURCE_PROXY__MAX_FILE_SIZE=50048576
## Size of chunks to read/write__
CKAN__RESOURCE_PROXY__CHUNK_SIZE=4096
## Default timeout for fetching proxied items
CKAN__RESOURCE_PROXY__TIMEOUT=10
CKAN__VIEWS__DEFAULT_VIEWS="image_view video_view audio_view webpage_view datatables_view text_view wmts_view geojson_view geo_view shp_view pdf_view"
# Localization
CKAN__LOCALE_DEFAULT="en"
CKAN__LOCALE_ORDER="en es pt_BR ja it cs_CZ ca fr el sv sr sr@latin no sk fi ru de pl nl bg ko_KR hu sa sl lv"
CKAN__LOCALES_OFFERED="en es pt_BR ja it cs_CZ ca fr el sv sr sr@latin no sk fi ru de pl nl bg ko_KR hu sa sl lv"
# Extensions
CKAN__PLUGINS="stats image_view video_view audio_view webpage_view text_view datatables_view resource_proxy geo_view geojson_view wmts_view shp_view dcat dcat_rdf_harvester dcat_json_interface dcat_json_harvester resourcedictionary xloader csw_harvester waf_harvester doc_harvester harvest spatial_metadata spatial_query spatial_harvest_metadata_api keycloak ckan_harvester pdf_view pages fluent envvars datastore d4science_theme" #doi spatial googleanalytics
#d4science schemingdcat schemingdcat_datasets schemingdcat_groups schemingdcat_organizations schemingdcat_ckan_harvester schemingdcat_xls_harvester
# ckanext-harvest
CKAN__HARVEST__MQ__TYPE=redis
CKAN__HARVEST__MQ__HOSTNAME=redis #localhost
CKAN__HARVEST__MQ__PORT=${REDIS_PORT}
CKAN__HARVEST__MQ__REDIS_DB=${REDIS_CKAN_DATABASE}
# Clean-up mechanism for the harvest log table. The default is 30 days.
CKAN__HARVEST__LOG_TIMEFRAME=40
CKAN__HARVEST__TIMEOUT = 240 #in minutes
# ckanext-googleanalytics
#CKANEXT__GOOGLEANALYTICS_ID=${ANALYTICS_ID}
#CKANEXT__GOOGLEANALYTICS_ID="UA-1010101-1"
#CKANEXT_GOOGLEANALYTICS_ID_FOR_ADMIN=UA-XXXXX-Z # Sostituisci con il tuo ID di monitoraggio di Google Analytics
#CKANEXT_GOOGLEANALYTICS_TRACK_EVENTS=true
#CKANEXT_GOOGLEANALYTICS_ANONYMIZE_IP=true
#DOI
CKANEXT__DOI__PREFIX=""
# Keycloak
CKANEXT__KEYCLOAK__SERVER_URL="https://accounts.d4science.org/auth/"
CKANEXT__KEYCLOAK__CLIENT_ID="d4s-catalogue-dev"
CKANEXT__KEYCLOAK__CLIENT_SECRET_KEY="ADD_THE_SECRET_KEY"
CKANEXT__KEYCLOAK__REALM_NAME="d4science"
CKANEXT__KEYCLOAK__REDIRECT_URI="https://localhost:8443/catalog/user/sso_login"
CKANEXT__KEYCLOAK__ENABLE_CKAN_INTERNAL_LOGIN=true
# ckanext-xloader
CKANEXT__XLOADER__JOBS__DB_URI=postgresql://${CKAN_DB_USER}:${CKAN_DB_PASSWORD}@${POSTGRES_HOST}/${CKAN_DB}
# ckanext-dcat
CKANEXT__DCAT__BASE_URI=${CKAN_URL}
# Default profile(s). Instead of this envvar, it's possible to specify all the profile(s) availables to be used for serialization using the profiles parameter: http://localhost:5000/catalog.xml?profiles=eu_dcat_ap_2,es_dcat
CKANEXT__DCAT__RDF_PROFILES='eu_dcat_ap_2 eu_dcat_ap_scheming'
# The custom endpoint **must** start with a forward slash (`/`) and contain the `{_format}` placeholder. The endpoint is added to the CKAN_SITE_URL and CKAN__ROOT_PATH, example: http://localhost:5000/catalog/catalog.rdf
CKANEXT__DCAT__DEFAULT_CATALOG_ENDPOINT='/catalog.{_format}'
# ckanext-spatial (Solr Backend - solr9-spatial: https://docs.ckan.org/projects/ckanext-spatial/en/latest/spatial-search.html#choosing-a-backend-for-the-spatial-search)
# ckanext-spatial (Solr Backend - solr9-spatial: https://docs.ckan.org/projects/ckanext-spatial/en/latest/spatial-search.html#choosing-a-backend-for-the-spatial-search)
CKANEXT__SPATIAL__SEARCH_BACKEND=solr-spatial-field
CKANEXT__SPATIAL__HARVEST__CONTINUE_ON_VALIDATION_ERRORS=true
CKAN__SPATIAL__SRID=3857
CKANEXT__SPATIAL__COMMON_MAP__TYPE=custom
CKANEXT__SPATIAL__COMMON_MAP__MAPBOX__ID=satellite-v9
CKANEXT__SPATIAL__COMMON_MAP__CUSTOM__URL=https://api.mapbox.com/styles/v1/mapbox/${CKANEXT__SPATIAL__COMMON_MAP__MAPBOX__ID}/tiles/256/{z}/{x}/{y}?access_token={access_token}
CKANEXT__SPATIAL__COMMON_MAP__ATTRIBUTION='© Mapbox <a href="https://www.mapbox.com/about/maps">Satellite</a>', <a href="https://openstreetmap.org">© OpenStreetMap</a>'
#CKANEXT__SPATIAL__COMMON_MAP__TYPE=custom
#CKANEXT__SPATIAL__COMMON_MAP__CUSTOM__URL=https://a.tile.openstreetmap.org/{z}/{x}/{y}.png
#CKANEXT__SPATIAL__COMMON_MAP__ATTRIBUTION='Map tiles by <a href="https://openstreetmap.org">OpenStreetMap</a> (<a href="https://creativecommons.org/licenses/by-sa/3.0">CC BY SA</a>)'
# ckanext-geoview
#CKANEXT__GEOVIEW__GEOJSON__MAX_FILE_SIZE=100000000
#CKANEXT__GEOVIEW__OL_VIEWER__FORMATS="wms wfs geojson gml kml"
#CKANEXT__GEOVIEW__SHP_VIEWER__SRID=3857
#CKANEXT__GEOVIEW__SHP_VIEWER__ENCODING=UTF-8
# ckanext-schemingdcat
## CSW Endpoint for spatial metadata
CKANEXT__SCHEMINGDCAT_GEOMETADATA_BASE_URI=${PYCSW_URL}
## Scheming: setup_scheming.sh
CKANEXT__SCHEMINGDCAT_DATASET_SCHEMA="ckanext.schemingdcat:schemas/geodcat_ap/eu_geodcat_ap_full.yaml"
CKANEXT__SCHEMINGDCAT_GROUP_SCHEMAS="ckanext.schemingdcat:schemas/geodcat_ap/eu_geodcat_ap_group.json"
CKANEXT__SCHEMINGDCAT_ORGANIZATION_SCHEMAS="ckanext.schemingdcat:schemas/geodcat_ap/eu_geodcat_ap_org.json"
CKANEXT__SCHEMINGDCAT_PRESETS="ckanext.schemingdcat:schemas/default_presets.json ckanext.fluent:presets.json"
## Facets: setup_scheming.sh
CKANEXT__SCHEMINGDCAT_FACET_LIST="dataset_scope theme groups theme_eu language dcat_type groups publisher_name publisher_type spatial_uri owner_org res_format frequency tags tag_uri conforms_to"
CKANEXT__SCHEMINGDCAT_ORGANIZATION_CUSTOM_FACETS=true
CKANEXT__SCHEMINGDCAT_GROUP_CUSTOM_FACETS=true
CKANEXT__SCHEMINGDCAT_DEFAULT_PACKAGE_ITEM_ICON="theme"
CKANEXT__SCHEMINGDCAT_DEFAULT_PACKAGE_ITEM_SHOW_SPATIAL=true
CKANEXT__SCHEMINGDCAT_SHOW_METADATA_TEMPLATES_TOOLBAR=false
CKANEXT__METADATA_TEMPLATES_SEARCH_IDENTIFIER="schemingdcat_xls-template"
CKANEXT__SCHEMINGDCAT_ENDPOINTS_YAML="endpoints.yaml"
CKANEXT__SCHEMINGDCAT__SOCIAL_GITHUB="https://github.com/mjanez/ckanext-schemingdcat"
CKANEXT__SCHEMINGDCAT__SOCIAL_X="https://x.com/ckanproject"
CKANEXT__SCHEMINGDCAT__SOCIAL_LINKEDIN="https://www.linkedin.com/company/ckanproject"
# ckanext-pages
CKANEXT__PAGES__ALOW_HTML=false
CKANEXT__PAGES__ORGANIZATION=true
CKANEXT__PAGES__GROUP=true
CKANEXT__PAGES__ABOUT_MENU=false
CKANEXT__PAGES__GROUP_MENU=true
CKANEXT__PAGES__ORGANIZATION_MENU=true
# ckan-pycsw
CSW_IDENTIFICATION_TITLE="Sample Geospatial Catalogue"
CSW_IDENTIFICATION_ABSTRACT="OGC CSW server powered by pycsw"
CSW_PROVIDER_NAME="ckan-docker development team"
CSW_PROVIDER_URL="https://github.com/mjanez/ckan-docker"
CSW_CONTACT_NAME="ckan-docker development team"
CSW_CONTACT_POSITION="Site Administrator"
CSW_CONTACT_ADDRESS="ckan-docker development team"
CSW_CONTACT_CITY="Madrid"
CSW_CONTACT_STATE_OR_PROVINCE="Madrid"
CSW_CONTACT_POSTAL_CODE="28001"
CSW_CONTACT_COUNTRY="Spain"
CSW_CONTACT_EMAIL=${CKAN_SYSADMIN_EMAIL}
CSW_CONTACT_URL=${CKAN_URL}
## Date of the last update of the metadata. ISO 8601 format): YYYY-MM-DD
CSW_INSPIRE_DATE="2024-01-01"
## Keywords separated by commas based on the GEMET Thesaurus keywordValue: https://inspire.ec.europa.eu/schemas/common/1.0/enums/enum_eng.xsd
CSW_INSPIRE_GEMET_KEYWORDS="Utility and governmental services"
# The level of INSPIRE conformance for spatial data sets and services (conformant, notConformant, notEvaluated)
CSW_INSPIRE_CONFORMITY="notEvaluated"
CSW_INSPIRE_CONTACT_NAME=${CSW_CONTACT_NAME}
CSW_INSPIRE_CONTACT_EMAIL=${CKAN_SYSADMIN_EMAIL}
# Temporal extent of the service (in ISO 8601 format). Either a single date (i.e. YYYY-MM-DD), or an extent (i.e. YYYY-MM-DD/YYYY-MM-DD)
CSW_INSPIRE_TEMP_EXTENT="2024-01-01"
##PRIVATE DATASET
CKAN__SEARCH__DEFAULT_INCLUDE_PRIVATE=true
CKAN__AUTH__REVEAL_PRIVATE_DATASETS=true
##FOCUSED DATASET TYPES <type>:<image_path/icon> images must be uploaded in /catalog/icons/tags-icons/
CKAN__FOCUSED_DATASET_TYPES="Dataset:database.svg, Service:service_tag.svg, Deliverable:papers-text.svg, Guidelines:note-list-check.svg, Poster:tv.svg, Presentation:presentation-text.svg, Provider:speedometer.svg, Method:bookmark-star.svg, Lesson:book-text.svg"
#get icons from https://fontawesome.com
CKAN__FOCUSED_DATASET_TYPES_ICONS="Dataset:fa-solid fa-database, Deliverable:fa-solid fa-newspaper, Guidelines:fa-solid fa-clipboard-list, Method:fa-solid fa-bookmark"
##RESOURCE-ACCESS
#the system_type key will always have priority
CKANEXT__D4SCIENCE_THEME__PUBLIC_DATASET__PUBLIC_ACCESS_TO_RESOURCES={"org_name1": True/False, "org_name2.system_type.value": True/False, "org_name3": True/False, "org_name3.system_type.value": True/False}
###AUTHORIZATION SETTINGS
CKAN__AUTH__ANON_CREATE_DATASET=false
CKAN__AUTH__CREATE_UNOWNED_DATASET=false
CKAN__AUTH__CREATE_DATASET_IF_NOT_IN_ORGANIZATION=false
CKAN__AUTH__USER_CREATE_GROUPS=false
CKAN__AUTH__USER_CREATE_ORGANIZATIONS=false
CKAN__AUTH__USER_DELETE_GROUPS=false
CKAN__AUTH__USER_DELETE_ORGANIZATIONS=false
CKAN__AUTH__CREATE_USER_VIA_API=false
CKAN__AUTH__CREATE_USER_VIA_WEB=false
CKAN__AUTH__ROLES_THAT_CASCADE_TO_SUB_GROUPS=admin
CKAN__AUTH__PUBLIC_USER_DETAILS=true
CKAN__AUTH__PUBLIC_ACTIVITY_STREAM_DETAIL=false
# Enables or disable collaborators in individual datasets (https://docs.ckan.org/en/2.9/maintaining/authorization.html#dataset-collaborators)
CKAN__AUTH__ALLOW_DATASET_COLLABORATORS=false
CKAN__AUTH__ALLOW_ADMIN_COLLABORATORS=false
CKAN__AUTH__ALLOW_COLLABORATORS_TO_CHANGE_OWNER_ORG=false
CKAN__AUTH__CREATE_DEFAULT_API_KEYS=true
CKAN__AUTH__ROUTE_AFTER_LOGIN="dashboard.datasets"

1
.gitattributes vendored
View File

@ -1 +0,0 @@
* text=auto eol=lf

61
.github/.old/build-master.yml vendored Normal file
View File

@ -0,0 +1,61 @@
name: Build CKAN Docker auxiliary images - master
on:
# Trigger the workflow on push or pull request
push:
branches: ['master']
pull_request:
branches:
- master
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
- name: NGINX build
uses: docker/build-push-action@v4
with:
context: ./nginx
file: ./nginx/Dockerfile
push: false
tags: mjanez/ckan-docker-nginx:test-build-only
- name: Apache HTTP Server build
uses: docker/build-push-action@v4
with:
context: ./apache
file: ./apache/Dockerfile
push: false
tags: mjanez/ckan-docker-apache:test-build-only
- name: PostgreSQL build
uses: docker/build-push-action@v4
with:
context: ./postgresql
file: ./postgresql/Dockerfile
push: false
tags: mjanez/ckan-docker-postgresql:test-build-only
- name: Solr build
uses: docker/build-push-action@v4
with:
context: ./solr
file: ./solr/Dockerfile
push: false
tags: mjanez/ckan-docker-solr:test-build-only
- name: ckan-pycsw build
uses: docker/build-push-action@v4
with:
context: ./ckan-pycsw
file: ./ckan-pycsw/Dockerfile
push: false
tags: mjanez/ckan-docker-pycsw:test-build-only

62
.github/.old/build-tags.yml vendored Normal file
View File

@ -0,0 +1,62 @@
name: Build CKAN Docker auxiliary images - ckan-*.*.*
on:
# Trigger the workflow on push or pull request
push:
branches:
- 'ckan-*.*.*'
pull_request:
branches:
- ckan-*.*.*
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
- name: NGINX build
uses: docker/build-push-action@v4
with:
context: ./nginx
file: ./nginx/Dockerfile
push: false
tags: mjanez/ckan-docker-nginx:test-build-only
- name: Apache HTTP Server build
uses: docker/build-push-action@v4
with:
context: ./apache
file: ./apache/Dockerfile
push: false
tags: mjanez/ckan-docker-apache:test-build-only
- name: PostgreSQL build
uses: docker/build-push-action@v4
with:
context: ./postgresql
file: ./postgresql/Dockerfile
push: false
tags: mjanez/ckan-docker-postgresql:test-build-only
- name: Solr build
uses: docker/build-push-action@v4
with:
context: ./solr
file: ./solr/Dockerfile
push: false
tags: mjanez/ckan-docker-solr:test-build-only
- name: ckan-pycsw build
uses: docker/build-push-action@v4
with:
context: ./ckan-pycsw
file: ./ckan-pycsw/Dockerfile
push: false
tags: mjanez/ckan-docker-pycsw:test-build-only

52
.github/.old/docker-master.yml vendored Normal file
View File

@ -0,0 +1,52 @@
name: Build & Push ckan-spatial - master
on:
# Trigger the workflow after build.yml
workflow_run:
workflows: ["Build CKAN Docker auxiliary images - master"]
branches: ['master']
pull_request:
branches: ['master']
types:
- completed
env:
REGISTRY: ghcr.io
IMAGE_NAME: mjanez/ckan-spatial
jobs:
build:
runs-on: ubuntu-latest
steps:
-
name: Set up QEMU
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Checkout
uses: actions/checkout@v4
-
name: Login to registry
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
-
name: Extract Docker metadata
id: meta
uses: docker/metadata-action@v4
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
-
name: Build and push
uses: docker/build-push-action@v5
with:
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
context: ./ckan
file: ./ckan/Dockerfile

49
.github/.old/docker-tags.yml vendored Normal file
View File

@ -0,0 +1,49 @@
name: Build & Push ckan-spatial - ckan-*.*.*
on:
push:
branches:
- 'ckan-*.*.*'
- '!dev/ckan-*.*.*'
env:
REGISTRY: ghcr.io
IMAGE_NAME: mjanez/ckan-spatial
jobs:
build:
runs-on: ubuntu-latest
steps:
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Checkout
uses: actions/checkout@v2
-
name: Login to registry
if: github.event_name != 'pull_request'
uses: docker/login-action@v2
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
-
name: Extract Docker metadata
id: meta
uses: docker/metadata-action@v4
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
-
name: Build and push
uses: docker/build-push-action@v3
with:
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
context: ./ckan
file: ./ckan/Dockerfile

View File

@ -1,48 +0,0 @@
name: Build CKAN Docker
on:
# Trigger the workflow on push or pull request,
# but only for the master branch
push:
branches:
- master
pull_request:
branches:
- master
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
- name: NGINX build
uses: docker/build-push-action@v2
with:
context: ./nginx
file: ./nginx/Dockerfile
push: false
tags: kowhai/ckan-docker-nginx:test-build-only
- name: PostgreSQL build
uses: docker/build-push-action@v2
with:
context: ./postgresql
file: ./postgresql/Dockerfile
push: false
tags: kowhai/ckan-docker-postgresql:test-build-only
- name: CKAN build
uses: docker/build-push-action@v2
with:
context: ./ckan
file: ./ckan/Dockerfile
push: false
tags: kowhai/ckan-docker-ckan:test-build-only

94
.github/workflows/docker-build.yml vendored Normal file
View File

@ -0,0 +1,94 @@
name: Build and push ckan-docker image from PR Merge
on:
pull_request:
types:
- closed
branches:
- master
- 'ckan-*.*.*'
- '!dev/ckan-*.*.*'
- '!feature/*'
- '!fix/*'
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
CONTEXT: .
BRANCH: ${{ github.head_ref }}
VERSION: ${{ github.head_ref }}
DOCKERFILE_PATH: /ckan
DOCKERFILE: Dockerfile
jobs:
docker:
name: runner/build-docker-push:${{ github.head_ref }}
runs-on: ubuntu-latest
if: github.event.pull_request.merged == true
steps:
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Check out code
uses: actions/checkout@v4
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract tag version from branch name
id: extract_tag_version
run: |
if [ "${{ env.BRANCH }}" = "master" ]; then
echo "VERSION=master" >> $GITHUB_ENV
else
echo "VERSION=$(echo ${{ env.BRANCH }} | sed 's/^ckan-//')" >> $GITHUB_ENV
fi
- name: Extract Docker metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
labels: |
org.opencontainers.image.documentation=https://github.com/${{ github.repository }}/blob/${{ env.BRANCH }}/README.md
org.opencontainers.image.version=${{ env.VERSION }}
annotations: |
org.opencontainers.image.description=This image contains CKAN based on a Docker Compose deployment. The container includes CKAN along with its dependencies and configurations for spatial data support.
org.opencontainers.image.source=https://github.com/${{ github.repository }}
- name: Build and push
uses: docker/build-push-action@v6
with:
push: true
tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.VERSION }}
labels: ${{ steps.meta.outputs.labels }}
annotations: ${{ steps.meta.outputs.annotations }}
context: ${{ env.CONTEXT }}${{ env.DOCKERFILE_PATH }}
file: ${{ env.CONTEXT }}${{ env.DOCKERFILE_PATH }}/${{ env.DOCKERFILE }}
- name: Linting Dockerfile with hadolint in GH Actions
uses: hadolint/hadolint-action@v3.1.0
with:
dockerfile: ${{ env.CONTEXT }}${{ env.DOCKERFILE_PATH }}/${{ env.DOCKERFILE }}
no-fail: true
- name: Run Trivy container image vulnerability scanner
uses: aquasecurity/trivy-action@0.24.0
with:
image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.VERSION }}
format: sarif
output: trivy-results.sarif
- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: trivy-results.sarif

84
.github/workflows/docker-manual.yml vendored Normal file
View File

@ -0,0 +1,84 @@
name: Build and release image manually
on: workflow_dispatch
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
CONTEXT: .
BRANCH: ${{ github.ref_name }}
VERSION: ${{ github.ref_name }}
DOCKERFILE_PATH: /ckan
DOCKERFILE: Dockerfile
jobs:
docker:
name: manual/build-docker-push:${{ github.ref_name }}
runs-on: ubuntu-latest
steps:
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Check out code
uses: actions/checkout@v4
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract tag version from branch name
id: extract_tag_version
run: |
if [ "${{ env.BRANCH }}" = "master" ]; then
echo "VERSION=master" >> $GITHUB_ENV
else
echo "VERSION=$(echo ${{ env.BRANCH }} | sed 's/^ckan-//')" >> $GITHUB_ENV
fi
- name: Extract Docker metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
labels: |
org.opencontainers.image.documentation=https://github.com/${{ github.repository }}/blob/${{ env.BRANCH }}/README.md
org.opencontainers.image.version=${{ env.VERSION }}
annotations: |
org.opencontainers.image.description=This image contains CKAN based on a Docker Compose deployment. The container includes CKAN along with its dependencies and configurations for spatial data support.
org.opencontainers.image.source=https://github.com/${{ github.repository }}
- name: Build and push
uses: docker/build-push-action@v6
with:
push: true
tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.VERSION }}
labels: ${{ steps.meta.outputs.labels }}
annotations: ${{ steps.meta.outputs.annotations }}
context: ${{ env.CONTEXT }}${{ env.DOCKERFILE_PATH }}
file: ${{ env.CONTEXT }}${{ env.DOCKERFILE_PATH }}/${{ env.DOCKERFILE }}
- name: Linting Dockerfile with hadolint in GH Actions
uses: hadolint/hadolint-action@v3.1.0
with:
dockerfile: ${{ env.CONTEXT }}${{ env.DOCKERFILE_PATH }}/${{ env.DOCKERFILE }}
no-fail: true
- name: Run Trivy container image vulnerability scanner
uses: aquasecurity/trivy-action@0.24.0
with:
image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.VERSION }}
format: sarif
output: trivy-results.sarif
- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: trivy-results.sarif

84
.github/workflows/docker-master.yml vendored Normal file
View File

@ -0,0 +1,84 @@
name: Build and push ckan-docker image from master push
on:
push:
branches:
- master
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
CONTEXT: .
BRANCH: master
VERSION: master
DOCKERFILE_PATH: /ckan
DOCKERFILE: Dockerfile
jobs:
docker:
name: runner/build-docker-push:master
runs-on: ubuntu-latest
steps:
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Check out code
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Get highest ckan branch excluding -dev
id: getbranch
run: echo "VERSION=$(git branch -r | grep -o 'ckan-[0-9]*\.[0-9]*\.[0-9]*[^-dev]$' | sort -V | tail -n 1)" >> $GITHUB_ENV
- name: Extract Docker metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
labels: |
org.opencontainers.image.documentation=https://github.com/${{ github.repository }}/blob/${{ env.BRANCH }}/README.md
org.opencontainers.image.version=${{ env.VERSION }}
annotations: |
org.opencontainers.image.description=This image contains CKAN based on a Docker Compose deployment. The container includes CKAN along with its dependencies and configurations for spatial data support.
org.opencontainers.image.source=https://github.com/${{ github.repository }}
- name: Build and push
uses: docker/build-push-action@v6
with:
push: true
tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.VERSION }}
labels: ${{ steps.meta.outputs.labels }}
annotations: ${{ steps.meta.outputs.annotations }}
context: ${{ env.CONTEXT }}${{ env.DOCKERFILE_PATH }}
file: ${{ env.CONTEXT }}${{ env.DOCKERFILE_PATH }}/${{ env.DOCKERFILE }}
- name: Linting Dockerfile with hadolint in GH Actions
uses: hadolint/hadolint-action@v3.1.0
with:
dockerfile: ${{ env.CONTEXT }}${{ env.DOCKERFILE_PATH }}/${{ env.DOCKERFILE }}
no-fail: true
- name: Run Trivy container image vulnerability scanner
uses: aquasecurity/trivy-action@0.24.0
with:
image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.VERSION }}
format: sarif
output: trivy-results.sarif
- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: trivy-results.sarif

89
.github/workflows/docker-pr.yml vendored Normal file
View File

@ -0,0 +1,89 @@
name: Test ckan-docker images (PR)
on:
pull_request:
branches:
- master
- 'ckan-*.*.*'
- '!dev/ckan-*.*.*'
- '!feature/*'
- '!fix/*'
env:
REGISTRY: ghcr.io
IMAGE_NAME: ckan-docker
CONTEXT: .
BRANCH: ${{ github.head_ref }}
DOCKERFILE_PATH: /ckan
DOCKERFILE: Dockerfile
HADOLINT_VERSION: 2.12.0
jobs:
docker:
name: runner/test-docker-pr:${{ github.head_ref }}
runs-on: ubuntu-latest
if: github.event_name == 'pull_request'
steps:
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Check out code
uses: actions/checkout@v4
- name: NGINX build
uses: docker/build-push-action@v6
with:
context: ./nginx
file: ./nginx/Dockerfile
push: false
tags: mjanez/ckan-docker-nginx:test-build-only
- name: Apache HTTP Server build
uses: docker/build-push-action@v6
with:
context: ./apache
file: ./apache/Dockerfile
push: false
tags: mjanez/ckan-docker-apache:test-build-only
- name: PostgreSQL build
uses: docker/build-push-action@v6
with:
context: ./postgresql
file: ./postgresql/Dockerfile
push: false
tags: mjanez/ckan-docker-postgresql:test-build-only
- name: Solr build
uses: docker/build-push-action@v6
with:
context: ./solr
file: ./solr/Dockerfile
push: false
tags: mjanez/ckan-docker-solr:test-build-only
- name: ckan-pycsw build
uses: docker/build-push-action@v6
with:
context: ./ckan-pycsw
file: ./ckan-pycsw/Dockerfile
push: false
tags: mjanez/ckan-docker-pycsw:test-build-only
- name: Extract Docker metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
- name: Linting Dockerfiles and annotate code inline in the github PR viewer
id: hadolint
uses: jbergstroem/hadolint-gh-action@v1.12.2
with:
dockerfile: ${{ env.CONTEXT }}${{ env.DOCKERFILE_PATH }}/${{ env.DOCKERFILE }} ${{ env.CONTEXT }}${{ env.DOCKERFILE_PATH }}/${{ env.DOCKERFILE }}.ghcr
version: ${{ env.HADOLINT_VERSION }}
annotate: true
error_level: -1

308
.gitignore vendored
View File

@ -1,285 +1,33 @@
# Created by https://www.toptal.com/developers/gitignore/api/python,visualstudiocode,pycharm+iml
# Edit at https://www.toptal.com/developers/gitignore?templates=python,visualstudiocode,pycharm+iml
# generic
.DS_Store
.vagrant
### PyCharm+iml ###
# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio, WebStorm and Rider
# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839
# code & config
_service-provider/*
_solr/schema.xml
_src/*
local/*
.vscode
__pycache__
log/*
!log/README.md
metadata/*
!metadata/README.md
src/*
!src/README.md
# User-specific stuff
.idea/**/workspace.xml
.idea/**/tasks.xml
.idea/**/usage.statistics.xml
.idea/**/dictionaries
.idea/**/shelf
# AWS User-specific
.idea/**/aws.xml
# Generated files
.idea/**/contentModel.xml
# Sensitive or high-churn files
.idea/**/dataSources/
.idea/**/dataSources.ids
.idea/**/dataSources.local.xml
.idea/**/sqlDataSources.xml
.idea/**/dynamic.xml
.idea/**/uiDesigner.xml
.idea/**/dbnavigator.xml
# Gradle
.idea/**/gradle.xml
.idea/**/libraries
# Gradle and Maven with auto-import
# When using Gradle or Maven with auto-import, you should exclude module files,
# since they will be recreated, and may cause churn. Uncomment if using
# auto-import.
# .idea/artifacts
# .idea/compiler.xml
# .idea/jarRepositories.xml
# .idea/modules.xml
# .idea/*.iml
# .idea/modules
# *.iml
# *.ipr
# CMake
cmake-build-*/
# Mongo Explorer plugin
.idea/**/mongoSettings.xml
# File-based project format
*.iws
# IntelliJ
out/
# mpeltonen/sbt-idea plugin
.idea_modules/
# JIRA plugin
atlassian-ide-plugin.xml
# Cursive Clojure plugin
.idea/replstate.xml
# SonarLint plugin
.idea/sonarlint/
# Crashlytics plugin (for Android Studio and IntelliJ)
com_crashlytics_export_strings.xml
crashlytics.properties
crashlytics-build.properties
fabric.properties
# Editor-based Rest Client
.idea/httpRequests
# Android studio 3.1+ serialized cache file
.idea/caches/build_file_checksums.ser
### PyCharm+iml Patch ###
# Reason: https://github.com/joeblau/gitignore.io/issues/186#issuecomment-249601023
*.iml
modules.xml
.idea/misc.xml
*.ipr
### Python ###
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
.pybuilder/
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock
# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
# in version control.
# https://pdm.fming.dev/#use-with-ide
.pdm.toml
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
# environment
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
.env.*
pycsw.conf.template.*
index.html.*
# Spyder project settings
.spyderproject
.spyproject
# environment allowlist
!.env.example
!.env.codespaces
!.env.es.example
!.env.dev.example
samples/api/*
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# pytype static type analyzer
.pytype/
# Cython debug symbols
cython_debug/
# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
### Python Patch ###
# Poetry local configuration file - https://python-poetry.org/docs/configuration/#local-configuration
poetry.toml
# ruff
.ruff_cache/
# LSP config files
pyrightconfig.json
### VisualStudioCode ###
.vscode/*
!.vscode/settings.json
!.vscode/tasks.json
!.vscode/launch.json
!.vscode/extensions.json
!.vscode/*.code-snippets
# Local History for Visual Studio Code
.history/
# Built Visual Studio Code Extensions
*.vsix
### VisualStudioCode Patch ###
# Ignore all local history of files
.history
.ionide
# End of https://www.toptal.com/developers/gitignore/api/python,visualstudiocode,pycharm+iml
.git-old/*
.github
# docker compose files
docker-compose.local-*.yml

3
.gitmodules vendored
View File

@ -1,3 +0,0 @@
[submodule "compose/2.11/src/ckanext-geonetwork"]
path = compose/2.11/src/ckanext-geonetwork
url = https://github.com/geosolutions-it/ckanext-geonetwork

View File

@ -1,3 +0,0 @@
ignored:
- DL3018
- DL3008

11
.project Normal file
View File

@ -0,0 +1,11 @@
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>docker-ckan</name>
<comment></comment>
<projects>
</projects>
<buildSpec>
</buildSpec>
<natures>
</natures>
</projectDescription>

View File

@ -1,8 +0,0 @@
{
"files.associations": {
"*html": "jinja-html"
},
"emmet.includeLanguages": {
"jinja-html": "*html"
}
}

View File

@ -1,76 +0,0 @@
# Contributor Covenant Code of Conduct
## Our Pledge
In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, sex characteristics, gender identity and expression,
level of experience, education, socio-economic status, nationality, personal
appearance, race, religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment
include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
## Scope
This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an appointed
representative at an online or offline event. Representation of a project may be
further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at info@keitaro.com. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
[homepage]: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see
https://www.contributor-covenant.org/faq

View File

@ -1,27 +0,0 @@
## How to contribute to Dockerized CKAN
#### **Did you find a bug?**
* **Ensure the bug was not already reported** by searching on GitHub under [Issues](https://github.com/keitaroinc/docker-ckan/issues).
* If you're unable to find an open issue addressing the problem, [open a new one](https://github.com/keitaroinc/docker-ckan/issues/new). Be sure to include a **title and clear description**, as much relevant information as possible, we include an issue template to help out in filling-in the issue.
#### **Did you write a patch that fixes a bug?**
* Open a new GitHub pull request with the patch.
* Ensure the PR description clearly describes the problem and solution. Include the relevant issue number if applicable.
#### **Do you intend to add a new feature or change an existing one?**
* [Create a new feature issue](https://github.com/keitaroinc/docker-ckan/issues/new) using the Feature Request template and describe your proposed changes
* Submit a pull request referring to the relevant feature issue/s
#### **Do you have questions about the source code?**
* Ask any question about how to use Dockerized CKAN in our [gitter chat](https://gitter.im/keitaroinc/docker-ckan).
Thanks!
Keitaro Team

84
Jenkinsfile vendored Normal file
View File

@ -0,0 +1,84 @@
pipeline {
agent {
label 'agent-docker-harbor'
}
environment {
projectname = "d4science-ckan"
hubname = "harbor.d4science.org"
registryUrl = "https://harbor.d4science.org"
registryCredential = 'harbor-d4science-ckan'
CKAN_REPO = "gitea@code-repo.d4science.org:D4ScienceCatalogue/docker-ckan.git"
//CATALOG RELATED VARIABLES
CKAN_THEME_BRANCH = "ckan-dev"
CKAN_IMAGE_BRANCH = "dev_ckan_2.10.5"
CKAN_IMAGE_TAG = "latest"
}
stages {
stage('Set IMAGE NAME') {
steps {
script {
env.CKAN_IMAGE_NAME = "${CKAN_IMAGE_BRANCH}-${env.CKAN_THEME_BRANCH}"
echo "IMAGE NAME = ${env.CKAN_IMAGE_NAME}"
}
}
}
stage('Build Docker Image') {
steps {
script {
sh """
docker build -t $hubname/$projectname/${env.CKAN_IMAGE_NAME}:${env.CKAN_IMAGE_TAG} ./ckan --build-arg CACHEBUST=\$(date +%s) --build-arg THEME_BRANCH=${env.CKAN_THEME_NAME}
"""
}
}
}
stage('Login Harbor') {
steps{
script {
withCredentials([usernamePassword(credentialsId: registryCredential, usernameVariable: 'HARBOR_USER', passwordVariable: 'HARBOR_PASS')]) {
sh 'docker login $registryUrl -u $HARBOR_USER -p $HARBOR_PASS'
}
}
}
}
stage('Push Image') {
steps{
script {
sh "docker push $hubname/$projectname/${env.CKAN_IMAGE_NAME}:${env.CKAN_IMAGE_TAG}"
}
}
}
}
// post-build actions
post {
success {
echo 'The docker pipeline worked!'
slackSend channel: '#notifications_jenkins_prod',
color: '#36a64f',
message: "SUCCESS: *${env.JOB_NAME}* build #${env.BUILD_NUMBER} completed successfully. URL: ${env.BUILD_URL}"
}
failure {
echo 'The docker pipeline has failed'
slackSend channel: '#notifications_jenkins_prod',
color: '#d90000',
message: "FAILURE: *${env.JOB_NAME}* build #${env.BUILD_NUMBER} failed. URL: ${env.BUILD_URL}"
}
aborted {
slackSend channel: '#notifications_jenkins_prod',
color: '#FFD700',
message: "ABORTED: *${env.JOB_NAME}* build #${env.BUILD_NUMBER} was aborted. URL: ${env.BUILD_URL}"
}
}
}

190
LICENSE
View File

@ -1,190 +0,0 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
Copyright 2020 Keitaro AB
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

288
LICENSE.md Normal file
View File

@ -0,0 +1,288 @@
# EUROPEAN UNION PUBLIC LICENCE V. 1.2 (EUPL V. 1.2)
**EUPL © the European Union 2007, 2016**
This European Union Public Licence (the ‘EUPL’) applies to the Work (as defined
below) which is provided under the terms of this Licence. Any use of the Work,
other than as authorised under this Licence is prohibited (to the extent such
use is covered by a right of the copyright holder of the Work).
The Work is provided under the terms of this Licence when the Licensor (as
defined below) has placed the following notice immediately following the
copyright notice for the Work:
***Licensed under the EUPL***
or has expressed by any other means his willingness to license under the EUPL.
## 1. Definitions
In this Licence, the following terms have the following meaning:
* **The Licence**: this Licence.
* **The Original Work**: the work or software distributed or communicated by the
Licensor under this Licence, available as Source Code and also as Executable
Code as the case may be.
* **Derivative Works**: the works or software that could be created by the
Licensee, based upon the Original Work or modifications thereof. This Licence
does not define the extent of modification or dependence on the Original Work
required in order to classify a work as a Derivative Work; this extent is
determined by copyright law applicable in the country mentioned in Article 15.
* **The Work**: the Original Work or its Derivative Works.
* **The Source Code**: the human-readable form of the Work which is the most
convenient for people to study and modify.
* **The Executable Code**: any code which has generally been compiled and which is
meant to be interpreted by a computer as a program.
* **The Licensor**: the natural or legal person that distributes or communicates
the Work under the Licence.
* **Contributor(s)**: any natural or legal person who modifies the Work under the
Licence, or otherwise contributes to the creation of a Derivative Work.
* **The Licensee** or **You**: any natural or legal person who makes any usage of
the Work under the terms of the Licence.
* **Distribution** or **Communication**: any act of selling, giving, lending,
renting, distributing, communicating, transmitting, or otherwise making
available, online or offline, copies of the Work or providing access to its
essential functionalities at the disposal of any other natural or legal
person.
## 2. Scope of the rights granted by the Licence
The Licensor hereby grants You a worldwide, royalty-free, non-exclusive,
sublicensable licence to do the following, for the duration of copyright vested
in the Original Work:
* use the Work in any circumstance and for all usage,
* reproduce the Work,
* modify the Work, and make Derivative Works based upon the Work,
* communicate to the public, including the right to make available or display
the Work or copies thereof to the public and perform publicly, as the case may
be, the Work,
* distribute the Work or copies thereof,
* lend and rent the Work or copies thereof,
* sublicense rights in the Work or copies thereof.
Those rights can be exercised on any media, supports and formats, whether now
known or later invented, as far as the applicable law permits so.
In the countries where moral rights apply, the Licensor waives his right to
exercise his moral right to the extent allowed by law in order to make effective
the licence of the economic rights here above listed.
The Licensor grants to the Licensee royalty-free, non-exclusive usage rights to
any patents held by the Licensor, to the extent necessary to make use of the
rights granted on the Work under this Licence.
## 3. Communication of the Source Code
The Licensor may provide the Work either in its Source Code form, or as
Executable Code. If the Work is provided as Executable Code, the Licensor
provides in addition a machine-readable copy of the Source Code of the Work
along with each copy of the Work that the Licensor distributes or indicates, in
a notice following the copyright notice attached to the Work, a repository where
the Source Code is easily and freely accessible for as long as the Licensor
continues to distribute or communicate the Work.
## 4. Limitations on copyright
Nothing in this Licence is intended to deprive the Licensee of the benefits from
any exception or limitation to the exclusive rights of the rights owners in the
Work, of the exhaustion of those rights or of other applicable limitations
thereto.
## 5. Obligations of the Licensee
The grant of the rights mentioned above is subject to some restrictions and
obligations imposed on the Licensee. Those obligations are the following:
**Attribution right**: The Licensee shall keep intact all copyright, patent or
trademarks notices and all notices that refer to the Licence and to the
disclaimer of warranties. The Licensee must include a copy of such notices and a
copy of the Licence with every copy of the Work he/she distributes or
communicates. The Licensee must cause any Derivative Work to carry prominent
notices stating that the Work has been modified and the date of modification.
**Copyleft clause**: If the Licensee distributes or communicates copies of the
Original Works or Derivative Works, this Distribution or Communication will be
done under the terms of this Licence or of a later version of this Licence
unless the Original Work is expressly distributed only under this version of the
Licence — for example by communicating **‘EUPL v. 1.2 only’**. The Licensee
(becoming Licensor) cannot offer or impose any additional terms or conditions on
the Work or Derivative Work that alter or restrict the terms of the Licence.
**Compatibility clause**: If the Licensee Distributes or Communicates Derivative
Works or copies thereof based upon both the Work and another work licensed under
a Compatible Licence, this Distribution or Communication can be done under the
terms of this Compatible Licence. For the sake of this clause, **‘Compatible
Licence’** refers to the licences listed in the appendix attached to this Licence.
Should the Licensee's obligations under the Compatible Licence conflict with
his/her obligations under this Licence, the obligations of the Compatible
Licence shall prevail.
**Provision of Source Code**: When distributing or communicating copies of the Work,
the Licensee will provide a machine-readable copy of the Source Code or indicate
a repository where this Source will be easily and freely available for as long
as the Licensee continues to distribute or communicate the Work.
**Legal Protection**: This Licence does not grant permission to use the trade names,
trademarks, service marks, or names of the Licensor, except as required for
reasonable and customary use in describing the origin of the Work and
reproducing the content of the copyright notice.
## 6. Chain of Authorship
The original Licensor warrants that the copyright in the Original Work granted
hereunder is owned by him/her or licensed to him/her and that he/she has the
power and authority to grant the Licence.
Each Contributor warrants that the copyright in the modifications he/she brings
to the Work are owned by him/her or licensed to him/her and that he/she has the
power and authority to grant the Licence.
Each time You accept the Licence, the original Licensor and subsequent
Contributors grant You a licence to their contributions to the Work, under the
terms of this Licence.
## 7. Disclaimer of Warranty
The Work is a work in progress, which is continuously improved by numerous
Contributors. It is not a finished work and may therefore contain defects or
‘bugs’ inherent to this type of development.
For the above reason, the Work is provided under the Licence on an ‘as is’ basis
and without warranties of any kind concerning the Work, including without
limitation merchantability, fitness for a particular purpose, absence of defects
or errors, accuracy, non-infringement of intellectual property rights other than
copyright as stated in Article 6 of this Licence.
This disclaimer of warranty is an essential part of the Licence and a condition
for the grant of any rights to the Work.
## 8. Disclaimer of Liability
Except in the cases of wilful misconduct or damages directly caused to natural
persons, the Licensor will in no event be liable for any direct or indirect,
material or moral, damages of any kind, arising out of the Licence or of the use
of the Work, including without limitation, damages for loss of goodwill, work
stoppage, computer failure or malfunction, loss of data or any commercial
damage, even if the Licensor has been advised of the possibility of such damage.
However, the Licensor will be liable under statutory product liability laws as
far such laws apply to the Work.
## 9. Additional agreements
While distributing the Work, You may choose to conclude an additional agreement,
defining obligations or services consistent with this Licence. However, if
accepting obligations, You may act only on your own behalf and on your sole
responsibility, not on behalf of the original Licensor or any other Contributor,
and only if You agree to indemnify, defend, and hold each Contributor harmless
for any liability incurred by, or claims asserted against such Contributor by
the fact You have accepted any warranty or additional liability.
## 10. Acceptance of the Licence
The provisions of this Licence can be accepted by clicking on an icon ‘I agree’
placed under the bottom of a window displaying the text of this Licence or by
affirming consent in any other similar way, in accordance with the rules of
applicable law. Clicking on that icon indicates your clear and irrevocable
acceptance of this Licence and all of its terms and conditions.
Similarly, you irrevocably accept this Licence and all of its terms and
conditions by exercising any rights granted to You by Article 2 of this Licence,
such as the use of the Work, the creation by You of a Derivative Work or the
Distribution or Communication by You of the Work or copies thereof.
## 11. Information to the public
In case of any Distribution or Communication of the Work by means of electronic
communication by You (for example, by offering to download the Work from a
remote location) the distribution channel or media (for example, a website) must
at least provide to the public the information requested by the applicable law
regarding the Licensor, the Licence and the way it may be accessible, concluded,
stored and reproduced by the Licensee.
## 12. Termination of the Licence
The Licence and the rights granted hereunder will terminate automatically upon
any breach by the Licensee of the terms of the Licence.
Such a termination will not terminate the licences of any person who has
received the Work from the Licensee under the Licence, provided such persons
remain in full compliance with the Licence.
## 13. Miscellaneous
Without prejudice of Article 9 above, the Licence represents the complete
agreement between the Parties as to the Work.
If any provision of the Licence is invalid or unenforceable under applicable
law, this will not affect the validity or enforceability of the Licence as a
whole. Such provision will be construed or reformed so as necessary to make it
valid and enforceable.
The European Commission may publish other linguistic versions or new versions of
this Licence or updated versions of the Appendix, so far this is required and
reasonable, without reducing the scope of the rights granted by the Licence. New
versions of the Licence will be published with a unique version number.
All linguistic versions of this Licence, approved by the European Commission,
have identical value. Parties can take advantage of the linguistic version of
their choice.
## 14. Jurisdiction
Without prejudice to specific agreement between parties,
* any litigation resulting from the interpretation of this License, arising
between the European Union institutions, bodies, offices or agencies, as a
Licensor, and any Licensee, will be subject to the jurisdiction of the Court
of Justice of the European Union, as laid down in article 272 of the Treaty on
the Functioning of the European Union,
* any litigation arising between other parties and resulting from the
interpretation of this License, will be subject to the exclusive jurisdiction
of the competent court where the Licensor resides or conducts its primary
business.
## 15. Applicable Law
Without prejudice to specific agreement between parties,
* this Licence shall be governed by the law of the European Union Member State
where the Licensor has his seat, resides or has his registered office,
* this licence shall be governed by Belgian law if the Licensor has no seat,
residence or registered office inside a European Union Member State.
## Appendix
**Compatible Licences** according to Article 5 EUPL are:
* GNU General Public License (GPL) v. 2, v. 3
* GNU Affero General Public License (AGPL) v. 3
* Open Software License (OSL) v. 2.1, v. 3.0
* Eclipse Public License (EPL) v. 1.0
* CeCILL v. 2.0, v. 2.1
* Mozilla Public Licence (MPL) v. 2
* GNU Lesser General Public Licence (LGPL) v. 2.1, v. 3
* Creative Commons Attribution-ShareAlike v. 3.0 Unported (CC BY-SA 3.0) for
works other than software
* European Union Public Licence (EUPL) v. 1.1, v. 1.2
* Québec Free and Open-Source Licence — Reciprocity (LiLiQ-R) or Strong
Reciprocity (LiLiQ-R+).
The European Commission may update this Appendix to later versions of the above
licences without producing a new version of the EUPL, as long as they provide
the rights granted in Article 2 of this Licence and protect the covered Source
Code from exclusive appropriation.
All other changes or additions to this Appendix require the production of a new
EUPL version.

View File

@ -1,14 +0,0 @@
VERSION := 2.11.0-focal
DOCKERHUB_NAMESPACE ?= d4science
IMAGE := ${DOCKERHUB_NAMESPACE}/ckan:${VERSION}
build:
DOCKER_DEFAULT_PLATFORM=linux/amd64 docker build -f images/ckan/2.11/Dockerfile.focal --tag ${IMAGE} images/ckan/2.11
up:
cd compose/2.11 && docker compose up -d
up_build:
cd compose/2.11 && docker compose up --build -d
push: build
docker push ${IMAGE}

2
NOTICE
View File

@ -1,2 +0,0 @@
docker-ckan
Copyright (c) 2016 Keitaro AB

351
README.md
View File

@ -1,113 +1,272 @@
# Dockerized CKAN
# D4Science CKAN 2.10 Docker Image
[![build-status](https://github.com/keitaroinc/docker-ckan/workflows/Docker%20Image%20Build/badge.svg?branch=master)](https://github.com/keitaroinc/docker-ckan/actions) [![License][]][1] [![Docker Pulls][]][2] [![Chat on Gitter][]][3]
This repository contains base docker images, examples and docker-compose used to build and run CKAN.
Docker-based deployment of CKAN 2.10.5 for the D4Science catalogue infrastructure
We build and publish docker images built using this repository to Dockerhub:
- [CKAN docker images](https://hub.docker.com/r/keitaro/ckan).
- [Datapusher docker images](https://hub.docker.com/r/keitaro/ckan-datapusher)
This deployment is based on the [mjanez/ckan-docker](https://github.com/mjanez/ckan-docker) project, adapted and extended for D4Science infrastructure requirements.
and Github Container Registry:
- [CKAN docker images on GHCR](https://github.com/orgs/keitaroinc/packages/container/package/ckan)
- [Datapusher docker images on GHCR](https://github.com/orgs/keitaroinc/packages/container/package/datapusher)
## Table of Contents
Looking to run CKAN on Kubernetes? Check out our [CKAN Helm Chart](https://github.com/keitaroinc/ckan-helm)!
- [Prerequisites](#prerequisites)
- [Installation](#installation)
- [Running the Services](#running-the-services)
- [Service Endpoints](#service-endpoints)
- [Testing the Installation](#testing-the-installation)
- [Harvesting External Catalogs](#harvesting-external-catalogs)
- [Configuration](#configuration)
- [Architecture](#architecture)
- [Troubleshooting](#troubleshooting)
- [References](#references)
## Overview
Images are provided in two flavors:
- [Alpine Linux](https://alpinelinux.org/) based images
- [Ubuntu Focal](https://ubuntu.com/) based images are the ones ending with `-focal` in the tag name
## Prerequisites
The Docker containers include only the required extensions to start a CKAN instance. The docker images are built using a multi-stage docker approach in order to produce slim production grade docker images with the right libraries and configuration. This multi-stage approach allows us to build python binary wheels in the build stages that later on we install in the main stage.
- **Docker** >= 20.10
- **Docker Compose** >= 2.0
- **Git**
- At least **4GB RAM** and **10GB disk space** available
Directory layout:
- [compose](./compose) - contains a docker-compose setup allowing users to spin up a CKAN setup easily using [docker-compose](https://docs.docker.com/compose/)
- [images](./images) - includes docker contexts for building all supported CKAN versions and datapusher
- [examples](./examples) - includes examples on how to extend the CKAN docker images and how to run them
## Installation
## Running CKAN using docker-compose
To start CKAN using docker-compose, simply change into the *compose* directory and run
```sh
make up
```
Check if CKAN was succesfuly started on http://localhost:5000.
1. Clone the repository:
```bash
git clone gitea@code-repo.d4science.org:D4ScienceCatalogue/docker-ckan.git
cd docker-ckan
```
### Configuration
In order to configure CKAN within docker-compose we use both build/up time variables loaded via the [.env](./compose/2.9/.env) file, and runtime variables loaded via the [.ckan-env](./compose/2.9/.ckan-env) file.
2. Create your environment configuration:
```bash
cp .env.example .env
```
Variables in the [.env](./compose/2.9/.env) file are loaded when running `docker-compose build` and `docker-compose up`, while variables in [.ckan-env](./compose/2.9/.ckan-env) file are used withing the CKAN container at runtime to configure CKAN and CKAN extensions using [ckanext-envvars](https://github.com/okfn/ckanext-envvars).
3. (Optional) Edit `.env` to customize your deployment settings (database credentials, ports, etc.)
## Extending CKAN docker images
Check some examples of extending CKAN docker images in the [examples](./examples) directory.
## Running/Stop the Services
We recommend to use a multi-stage approach to extend the docker images that we provide here. To extend the images the following Dockerfile structure is recommended:
```docker
###################
### Extensions ####
###################
FROM ghcr.io/keitaroinc/ckan:2.9.9 as extbuild
### Development Environment
# Switch to the root user
USER root
# Install any system packages necessary to build extensions
# Make sure we install python 3.8, cause CKAN is not compatible with 3.9
RUN apk add --no-cache --repository=http://dl-cdn.alpinelinux.org/alpine/v3.13/main \
python3-dev=3.8.10-r0
# Fetch and build the custom CKAN extensions
RUN pip wheel --wheel-dir=/wheels git+https://github.com/acmecorp/ckanext-acme@0.0.1#egg=ckanext-acme
############
### MAIN ###
############
FROM ghcr.io/keitaroinc/ckan:2.9.9
# Add the custom extensions to the plugins list
ENV CKAN__PLUGINS envvars image_view text_view recline_view datastore datapusher acme
# Switch to the root user
USER root
COPY --from=extbuild /wheels /srv/app/ext_wheels
# Install and enable the custom extensions
RUN pip install --no-index --find-links=/srv/app/ext_wheels ckanext-acme && \
ckan config-tool ${APP_DIR}/production.ini "ckan.plugins = ${CKAN__PLUGINS}" && \
chown -R ckan:ckan /srv/app
# Remove wheels
RUN rm -rf /srv/app/ext_wheels
# Switch to the ckan user
USER ckan
**Linux:**
```bash
docker compose -f docker-compose.dev.yml up -d --build
```
### Adding init and afterinit scripts
You can add scripts to CKAN custom images and copy them to the *docker-entrypoint.d* directory. Any *.sh or *.py file in that directory will be executed before the main initialization script (prerun.py) is executed.
You can add scripts to CKAN custom images and copy them to the *docker-afterinit.d* directory. Any *.sh or *.py file in that directory will be executed after the main initialization script (prerun.py) is executed.
## Build
To build a CKAN image run:
```sh
make build
```
In the makefile, The `–-tag d4science-ckan:2.10.4-focal` flag sets the image name to d4science-ckan:2.10.4-focal and 'images/ckan/2.10' at the end tells docker build to use the context into the specified directory where the Dockerfile and related contents are.
## Upload to DockerHub
>*It's recommended to upload built images to DockerHub*
To upload the image to DockerHub run:
```sh
docker push [options] <docker-hub-namespace>/ckan:<image-tag>
```bash
sudo docker compose -f docker-compose.dev.yml down
```
[License]: https://img.shields.io/badge/license-Apache--2.0-blue.svg?style=flat
[1]: https://opensource.org/licenses/Apache-2.0
[Docker Pulls]: https://img.shields.io/docker/pulls/keitaro/ckan.svg?style=flat
[2]: https://hub.docker.com/r/keitaro/ckan
[Chat on Gitter]: https://badges.gitter.im/gitterHQ/gitter.svg
[3]: https://gitter.im/keitaroinc/docker-ckan
**macOS (Intel/Apple Silicon):**
```bash
DOCKER_DEFAULT_PLATFORM=linux/amd64 docker compose -f docker-compose.dev.yml up -d --build
```
```bash
DOCKER_DEFAULT_PLATFORM=linux/amd64 docker compose -f docker-compose.dev.yml down
```
### Production Environment
**Linux:**
```bash
docker compose up -d --build
```
**macOS:**
```bash
DOCKER_DEFAULT_PLATFORM=linux/amd64 docker compose up -d --build
```
### Monitoring Logs
To monitor the services in real-time:
```bash
docker compose logs -f ckan
```
To view all service logs:
```bash
docker compose logs -f
```
## Service Endpoints
By default, the following services are exposed:
| Service | URL | Description |
|---------|-----|-------------|
| **CKAN (via nginx)** | http://localhost:81/catalog | Main CKAN web interface |
| **CKAN (direct)** | http://localhost:5000 | Direct CKAN application (dev only) |
| **NGINX (HTTP)** | http://localhost:81 | Reverse proxy |
| **NGINX (HTTPS)** | https://localhost:8443 | Secure reverse proxy |
| **Solr** | http://localhost:8983 | Search engine admin interface |
> **Note:** Ports can be customized in the `.env` file.
## Testing the Installation
### 1. Access CKAN
Navigate to: http://localhost:81/catalog
### 2. Default Credentials
For testing purposes, use:
- **Username:** `ckan_admin`
- **Password:** `test1234`
> ⚠️ **Important:** Change these credentials in production environments!
### 3. Test Other Services
- **Solr Admin UI:** http://localhost:8983/solr
## Harvesting External Catalogs
To harvest metadata from external CKAN instances:
1. **Create an Organization**
- Navigate to: http://localhost:81/catalog/organization/
- Go to: **Organizations** → **Add Organization**
- Fill in the organization details and save
2. **Configure a Harvest Source**
- Navigate to: http://localhost:81/catalog/harvest/new
- Fill in all required fields:
- **URL:** The external CKAN instance URL (e.g., `https://ckan-sobigdata.d4science.org/`)
- **Source Type:** Select `CKAN`
- **Frequency:** Select `Manually` for testing
- **Organization:** Select the organization created in step 1
- Save the harvest source
3. **Run the Harvest**
- Navigate to the harvest source admin page
- Click **"Admin"** and then **"Reharvest"** to manually trigger the harvest
4. **Monitor Harvest Jobs**
- Check the harvest job status and logs in the admin interface
- Review harvested datasets in your organization
## Configuration
### Environment Variables
Key configuration options in `.env`:
```bash
# CKAN Configuration
CKAN_SITE_URL=http://localhost:81/catalog
CKAN_SYSADMIN_NAME=ckan_admin
CKAN_SYSADMIN_PASSWORD=test1234
CKAN_SYSADMIN_EMAIL=admin@example.com
# Startup Options
CKAN_SKIP_REINDEX_ON_STARTUP=true # Skip Solr index clear/rebuild on container startup (default: true)
# Database Configuration
POSTGRES_PASSWORD=ckan
CKAN_DB_USER=ckan_default
CKAN_DB_PASSWORD=ckan
# Ports
NGINX_PORT_HOST=81
NGINX_SSLPORT_HOST=8443
# Theme Branch (D4Science customization)
THEME_BRANCH=main
```
### Custom Extensions
This deployment includes the following CKAN extensions:
- **ckanext-harvest** - Metadata harvesting framework
- **ckanext-spatial** - Spatial metadata and search
- **ckanext-dcat** - DCAT/DCAT-AP metadata support
- **ckanext-xloader** - Fast data loading
- **ckanext-scheming** - Custom metadata schemas
- **ckanext-pages** - Static pages management
- **ckanext-hierarchy** - Organization hierarchies
- **ckanext-d4science_theme** - D4Science custom theme
- And more (see `Dockerfile` for the complete list)
## Architecture
```mermaid
graph TD
Browser[Browser] -->|HTTP/HTTPS| NGINX[NGINX<br/>Reverse Proxy]
NGINX --> CKAN[CKAN]
CKAN --> Solr[Solr]
CKAN --> Redis[Redis]
CKAN --> PostgreSQL[PostgreSQL]
CKAN --> Volumes[Volumes]
```
## Troubleshooting
### Container Won't Start
Check logs:
```bash
docker compose logs ckan
```
Verify all services are running:
```bash
docker compose ps
```
### Permission Issues
If you encounter permission errors related to i18n files:
```bash
docker exec -it <ckan_container_name> chown -R ckan:ckan /srv/app/src/ckan/ckan/i18n/
```
### Database Connection Issues
Ensure PostgreSQL is fully initialized:
```bash
docker compose logs db
```
Reset the database (⚠️ destroys all data):
```bash
docker compose down -v
docker compose up -d
```
### Port Conflicts
If ports are already in use, modify them in `.env`:
```bash
NGINX_PORT_HOST=8081
NGINX_SSLPORT_HOST=8444
```
### Rebuild Images
To rebuild images from scratch:
```bash
docker compose down
docker compose build --no-cache
docker compose up -d
```
## Authors
* **Francesco Mangiacrapa** ([ORCID](https://orcid.org/0000-0002-6528-664X)) Computer Scientist at [ISTI-CNR Infrascience Group](https://infrascience.isti.cnr.it/)
* **Marco Procaccini** Computer Scientist at [IGG-CNR](https://www.igg.cnr.it/)
## References
- **Original Docker Setup:** https://github.com/mjanez/ckan-docker
- **CKAN Documentation:** https://docs.ckan.org/
- **CKAN Extensions:** https://extensions.ckan.org/
- **D4Science Infrastructure:** https://www.d4science.org/
## License
This project is licensed under the terms specified in the [LICENSE.md](LICENSE.md) file.
## Support
For issues related to D4Science infrastructure, contact the D4Science catalogue team.

24
apache/Dockerfile Normal file
View File

@ -0,0 +1,24 @@
FROM httpd:2.4-alpine
ENV CKAN_PORT=5000
ENV PYCSW_PORT=8000
ENV PYCSW_COMPOSE_SERVICE=pycsw
ENV CKAN_COMPOSE_SERVICE=ckan
ENV CKAN_DEV_COMPOSE_SERVICE=ckan-dev
ENV APACHE_PORT=80
ENV APACHE_LOG_DIR=/var/log/apache
ENV APACHE_ROOT=/usr/local/apache2
ENV APACHE_DOCUMENT_ROOT=${APACHE_ROOT}/htdocs
ENV PROXY_SERVER_NAME=localhost
ENV PROXY_CKAN_LOCATION=/catalog
ENV PROXY_PYCSW_LOCATION=/csw
ENV PROXY_PYCSW_PROXY_PASS=http://${PYCSW_COMPOSE_SERVICE}:${PYCSW_PORT}
ENV PROXY_CKAN_PROXY_PASS=http://${CKAN_COMPOSE_SERVICE}:${CKAN_PORT}
RUN mkdir -p ${APACHE_LOG_DIR}
COPY setup/httpd.conf ${APACHE_ROOT}/conf/httpd.conf
COPY setup/index.html ${APACHE_DOCUMENT_ROOT}/index.html
EXPOSE ${APACHE_PORT}

570
apache/setup/httpd.conf Normal file
View File

@ -0,0 +1,570 @@
#
# This is the main Apache HTTP server configuration file. It contains the
# configuration directives that give the server its instructions.
# See <URL:http://httpd.apache.org/docs/2.4/> for detailed information.
# In particular, see
# <URL:http://httpd.apache.org/docs/2.4/mod/directives.html>
# for a discussion of each configuration directive.
#
# Do NOT simply read the instructions in here without understanding
# what they do. They're here only as hints or reminders. If you are unsure
# consult the online docs. You have been warned.
#
# Configuration and logfile names: If the filenames you specify for many
# of the server's control files begin with "/" (or "drive:/" for Win32), the
# server will use that explicit path. If the filenames do *not* begin
# with "/", the value of ServerRoot is prepended -- so "logs/access_log"
# with ServerRoot set to "/usr/local/apache2" will be interpreted by the
# server as "/usr/local/apache2/logs/access_log", whereas "/logs/access_log"
# will be interpreted as '/logs/access_log'.
#
# ServerRoot: The top of the directory tree under which the server's
# configuration, error, and log files are kept.
#
# Do not add a slash at the end of the directory path. If you point
# ServerRoot at a non-local disk, be sure to specify a local disk on the
# Mutex directive, if file-based mutexes are used. If you wish to share the
# same ServerRoot for multiple httpd daemons, you will need to change at
# least PidFile.
#
ServerRoot ${APACHE_ROOT}
#
# Mutex: Allows you to set the mutex mechanism and mutex file directory
# for individual mutexes, or change the global defaults
#
# Uncomment and change the directory if mutexes are file-based and the default
# mutex file directory is not on a local disk or is not appropriate for some
# other reason.
#
# Mutex default:logs
#
# Listen: Allows you to bind Apache to specific IP addresses and/or
# ports, instead of the default. See also the <VirtualHost>
# directive.
#
# Change this to Listen on specific IP addresses as shown below to
# prevent Apache from glomming onto all bound IP addresses.
#
#Listen 12.34.56.78:80
Listen ${APACHE_PORT}
#
# Dynamic Shared Object (DSO) Support
#
# To be able to use the functionality of a module which was built as a DSO you
# have to place corresponding `LoadModule' lines at this location so the
# directives contained in it are actually available _before_ they are used.
# Statically compiled modules (those listed by `httpd -l') do not need
# to be loaded here.
#
# Example:
# LoadModule foo_module modules/mod_foo.so
#
LoadModule mpm_event_module modules/mod_mpm_event.so
#LoadModule mpm_prefork_module modules/mod_mpm_prefork.so
#LoadModule mpm_worker_module modules/mod_mpm_worker.so
LoadModule authn_file_module modules/mod_authn_file.so
#LoadModule authn_dbm_module modules/mod_authn_dbm.so
#LoadModule authn_anon_module modules/mod_authn_anon.so
#LoadModule authn_dbd_module modules/mod_authn_dbd.so
#LoadModule authn_socache_module modules/mod_authn_socache.so
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
LoadModule authz_user_module modules/mod_authz_user.so
#LoadModule authz_dbm_module modules/mod_authz_dbm.so
#LoadModule authz_owner_module modules/mod_authz_owner.so
#LoadModule authz_dbd_module modules/mod_authz_dbd.so
LoadModule authz_core_module modules/mod_authz_core.so
#LoadModule authnz_ldap_module modules/mod_authnz_ldap.so
#LoadModule authnz_fcgi_module modules/mod_authnz_fcgi.so
LoadModule access_compat_module modules/mod_access_compat.so
LoadModule auth_basic_module modules/mod_auth_basic.so
#LoadModule auth_form_module modules/mod_auth_form.so
#LoadModule auth_digest_module modules/mod_auth_digest.so
#LoadModule allowmethods_module modules/mod_allowmethods.so
#LoadModule isapi_module modules/mod_isapi.so
#LoadModule file_cache_module modules/mod_file_cache.so
#LoadModule cache_module modules/mod_cache.so
#LoadModule cache_disk_module modules/mod_cache_disk.so
#LoadModule cache_socache_module modules/mod_cache_socache.so
#LoadModule socache_shmcb_module modules/mod_socache_shmcb.so
#LoadModule socache_dbm_module modules/mod_socache_dbm.so
#LoadModule socache_memcache_module modules/mod_socache_memcache.so
#LoadModule socache_redis_module modules/mod_socache_redis.so
#LoadModule watchdog_module modules/mod_watchdog.so
#LoadModule macro_module modules/mod_macro.so
#LoadModule dbd_module modules/mod_dbd.so
#LoadModule bucketeer_module modules/mod_bucketeer.so
#LoadModule dumpio_module modules/mod_dumpio.so
#LoadModule echo_module modules/mod_echo.so
#LoadModule example_hooks_module modules/mod_example_hooks.so
#LoadModule case_filter_module modules/mod_case_filter.so
#LoadModule case_filter_in_module modules/mod_case_filter_in.so
#LoadModule example_ipc_module modules/mod_example_ipc.so
#LoadModule buffer_module modules/mod_buffer.so
#LoadModule data_module modules/mod_data.so
#LoadModule ratelimit_module modules/mod_ratelimit.so
LoadModule reqtimeout_module modules/mod_reqtimeout.so
#LoadModule ext_filter_module modules/mod_ext_filter.so
#LoadModule request_module modules/mod_request.so
#LoadModule include_module modules/mod_include.so
LoadModule filter_module modules/mod_filter.so
#LoadModule reflector_module modules/mod_reflector.so
#LoadModule substitute_module modules/mod_substitute.so
#LoadModule sed_module modules/mod_sed.so
#LoadModule charset_lite_module modules/mod_charset_lite.so
#LoadModule deflate_module modules/mod_deflate.so
LoadModule xml2enc_module modules/mod_xml2enc.so
LoadModule proxy_html_module modules/mod_proxy_html.so
#LoadModule brotli_module modules/mod_brotli.so
LoadModule mime_module modules/mod_mime.so
#LoadModule ldap_module modules/mod_ldap.so
LoadModule log_config_module modules/mod_log_config.so
#LoadModule log_debug_module modules/mod_log_debug.so
#LoadModule log_forensic_module modules/mod_log_forensic.so
#LoadModule logio_module modules/mod_logio.so
#LoadModule lua_module modules/mod_lua.so
LoadModule env_module modules/mod_env.so
#LoadModule mime_magic_module modules/mod_mime_magic.so
#LoadModule cern_meta_module modules/mod_cern_meta.so
#LoadModule expires_module modules/mod_expires.so
LoadModule headers_module modules/mod_headers.so
#LoadModule ident_module modules/mod_ident.so
#LoadModule usertrack_module modules/mod_usertrack.so
#LoadModule unique_id_module modules/mod_unique_id.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule version_module modules/mod_version.so
#LoadModule remoteip_module modules/mod_remoteip.so
LoadModule proxy_module modules/mod_proxy.so
#LoadModule proxy_connect_module modules/mod_proxy_connect.so
#LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
#LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so
#LoadModule proxy_scgi_module modules/mod_proxy_scgi.so
#LoadModule proxy_uwsgi_module modules/mod_proxy_uwsgi.so
#LoadModule proxy_fdpass_module modules/mod_proxy_fdpass.so
#LoadModule proxy_wstunnel_module modules/mod_proxy_wstunnel.so
#LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
#LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
#LoadModule proxy_express_module modules/mod_proxy_express.so
#LoadModule proxy_hcheck_module modules/mod_proxy_hcheck.so
#LoadModule session_module modules/mod_session.so
#LoadModule session_cookie_module modules/mod_session_cookie.so
#LoadModule session_crypto_module modules/mod_session_crypto.so
#LoadModule session_dbd_module modules/mod_session_dbd.so
#LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
#LoadModule slotmem_plain_module modules/mod_slotmem_plain.so
#LoadModule ssl_module modules/mod_ssl.so
#LoadModule optional_hook_export_module modules/mod_optional_hook_export.so
#LoadModule optional_hook_import_module modules/mod_optional_hook_import.so
#LoadModule optional_fn_import_module modules/mod_optional_fn_import.so
#LoadModule optional_fn_export_module modules/mod_optional_fn_export.so
#LoadModule dialup_module modules/mod_dialup.so
LoadModule http2_module modules/mod_http2.so
LoadModule proxy_http2_module modules/mod_proxy_http2.so
#LoadModule md_module modules/mod_md.so
#LoadModule lbmethod_byrequests_module modules/mod_lbmethod_byrequests.so
#LoadModule lbmethod_bytraffic_module modules/mod_lbmethod_bytraffic.so
#LoadModule lbmethod_bybusyness_module modules/mod_lbmethod_bybusyness.so
#LoadModule lbmethod_heartbeat_module modules/mod_lbmethod_heartbeat.so
LoadModule unixd_module modules/mod_unixd.so
#LoadModule heartbeat_module modules/mod_heartbeat.so
#LoadModule heartmonitor_module modules/mod_heartmonitor.so
#LoadModule dav_module modules/mod_dav.so
LoadModule status_module modules/mod_status.so
LoadModule autoindex_module modules/mod_autoindex.so
#LoadModule asis_module modules/mod_asis.so
#LoadModule info_module modules/mod_info.so
#LoadModule suexec_module modules/mod_suexec.so
<IfModule !mpm_prefork_module>
#LoadModule cgid_module modules/mod_cgid.so
</IfModule>
<IfModule mpm_prefork_module>
#LoadModule cgi_module modules/mod_cgi.so
</IfModule>
#LoadModule dav_fs_module modules/mod_dav_fs.so
#LoadModule dav_lock_module modules/mod_dav_lock.so
#LoadModule vhost_alias_module modules/mod_vhost_alias.so
#LoadModule negotiation_module modules/mod_negotiation.so
LoadModule dir_module modules/mod_dir.so
#LoadModule imagemap_module modules/mod_imagemap.so
#LoadModule actions_module modules/mod_actions.so
#LoadModule speling_module modules/mod_speling.so
#LoadModule userdir_module modules/mod_userdir.so
LoadModule alias_module modules/mod_alias.so
#LoadModule rewrite_module modules/mod_rewrite.so
<IfModule unixd_module>
#
# If you wish httpd to run as a different user or group, you must run
# httpd as root initially and it will switch.
#
# User/Group: The name (or #number) of the user/group to run httpd as.
# It is usually good practice to create a dedicated user and group for
# running httpd, as with most system services.
#
User www-data
Group www-data
</IfModule>
# 'Main' server configuration
#
# The directives in this section set up the values used by the 'main'
# server, which responds to any requests that aren't handled by a
# <VirtualHost> definition. These values also provide defaults for
# any <VirtualHost> containers you may define later in the file.
#
# All of these directives may appear inside <VirtualHost> containers,
# in which case these default settings will be overridden for the
# virtual host being defined.
#
<VirtualHost *:${APACHE_PORT}>
<Location ${PROXY_CKAN_LOCATION}>
ProxyPass ${PROXY_CKAN_PROXY_PASS}
ProxyPassReverse ${PROXY_CKAN_PROXY_PASS}
</Location>
<Location ${PROXY_PYCSW_LOCATION}>
ProxyPass ${PROXY_PYCSW_PROXY_PASS}
ProxyPassReverse ${PROXY_PYCSW_PROXY_PASS}
</Location>
Protocols h2 http/1.1
ServerName ${PROXY_SERVER_NAME}
ServerAdmin webmaster@localhost
DocumentRoot ${APACHE_DOCUMENT_ROOT}
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>
#
# ServerAdmin: Your address, where problems with the server should be
# e-mailed. This address appears on some server-generated pages, such
# as error documents. e.g. admin@your-domain.com
#
ServerAdmin you@example.com
#
# ServerName gives the name and port that the server uses to identify itself.
# This can often be determined automatically, but we recommend you specify
# it explicitly to prevent problems during startup.
#
# If your host doesn't have a registered DNS name, enter its IP address here.
#
#ServerName www.example.com:80
#
# Deny access to the entirety of your server's filesystem. You must
# explicitly permit access to web content directories in other
# <Directory> blocks below.
#
<Directory />
AllowOverride none
Require all denied
</Directory>
#
# Note that from this point forward you must specifically allow
# particular features to be enabled - so if something's not working as
# you might expect, make sure that you have specifically enabled it
# below.
#
#
# DocumentRoot: The directory out of which you will serve your
# documents. By default, all requests are taken from this directory, but
# symbolic links and aliases may be used to point to other locations.
#
DocumentRoot ${APACHE_DOCUMENT_ROOT}
<Directory ${APACHE_DOCUMENT_ROOT}>
#
# Possible values for the Options directive are "None", "All",
# or any combination of:
# Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
#
# Note that "MultiViews" must be named *explicitly* --- "Options All"
# doesn't give it to you.
#
# The Options directive is both complicated and important. Please see
# http://httpd.apache.org/docs/2.4/mod/core.html#options
# for more information.
#
Options Indexes FollowSymLinks
#
# AllowOverride controls what directives may be placed in .htaccess files.
# It can be "All", "None", or any combination of the keywords:
# AllowOverride FileInfo AuthConfig Limit
#
AllowOverride None
#
# Controls who can get stuff from this server.
#
Require all granted
</Directory>
#
# DirectoryIndex: sets the file that Apache will serve if a directory
# is requested.
#
<IfModule dir_module>
DirectoryIndex index.html
</IfModule>
#
# The following lines prevent .htaccess and .htpasswd files from being
# viewed by Web clients.
#
<Files ".ht*">
Require all denied
</Files>
#
# ErrorLog: The location of the error log file.
# If you do not specify an ErrorLog directive within a <VirtualHost>
# container, error messages relating to that virtual host will be
# logged here. If you *do* define an error logfile for a <VirtualHost>
# container, that host's errors will be logged there and not here.
#
ErrorLog /proc/self/fd/2
#
# LogLevel: Control the number of messages logged to the error_log.
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
#
LogLevel warn
<IfModule log_config_module>
#
# The following directives define some format nicknames for use with
# a CustomLog directive (see below).
#
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
<IfModule logio_module>
# You need to enable mod_logio.c to use %I and %O
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
</IfModule>
#
# The location and format of the access logfile (Common Logfile Format).
# If you do not define any access logfiles within a <VirtualHost>
# container, they will be logged here. Contrariwise, if you *do*
# define per-<VirtualHost> access logfiles, transactions will be
# logged therein and *not* in this file.
#
CustomLog /proc/self/fd/1 common
#
# If you prefer a logfile with access, agent, and referer information
# (Combined Logfile Format) you can use the following directive.
#
#CustomLog "logs/access_log" combined
</IfModule>
<IfModule alias_module>
#
# Redirect: Allows you to tell clients about documents that used to
# exist in your server's namespace, but do not anymore. The client
# will make a new request for the document at its new location.
# Example:
# Redirect permanent /foo http://www.example.com/bar
#
# Alias: Maps web paths into filesystem paths and is used to
# access content that does not live under the DocumentRoot.
# Example:
# Alias /webpath /full/filesystem/path
#
# If you include a trailing / on /webpath then the server will
# require it to be present in the URL. You will also likely
# need to provide a <Directory> section to allow access to
# the filesystem path.
#
# ScriptAlias: This controls which directories contain server scripts.
# ScriptAliases are essentially the same as Aliases, except that
# documents in the target directory are treated as applications and
# run by the server when requested rather than as documents sent to the
# client. The same rules about trailing "/" apply to ScriptAlias
# directives as to Alias.
#
ScriptAlias /cgi-bin/ ${APACHE_ROOT}/cgi-bin/
</IfModule>
<IfModule cgid_module>
#
# ScriptSock: On threaded servers, designate the path to the UNIX
# socket used to communicate with the CGI daemon of mod_cgid.
#
#Scriptsock cgisock
</IfModule>
#
# "/usr/local/apache2/cgi-bin" should be changed to whatever your ScriptAliased
# CGI directory exists, if you have that configured.
#
<Directory ${APACHE_ROOT}/cgi-bin>
AllowOverride None
Options None
Require all granted
</Directory>
<IfModule headers_module>
#
# Avoid passing HTTP_PROXY environment to CGI's on this or any proxied
# backend servers which have lingering "httpoxy" defects.
# 'Proxy' request header is undefined by the IETF, not listed by IANA
#
RequestHeader unset Proxy early
</IfModule>
<IfModule mime_module>
#
# TypesConfig points to the file containing the list of mappings from
# filename extension to MIME-type.
#
TypesConfig conf/mime.types
#
# AddType allows you to add to or override the MIME configuration
# file specified in TypesConfig for specific file types.
#
#AddType application/x-gzip .tgz
#
# AddEncoding allows you to have certain browsers uncompress
# information on the fly. Note: Not all browsers support this.
#
#AddEncoding x-compress .Z
#AddEncoding x-gzip .gz .tgz
#
# If the AddEncoding directives above are commented-out, then you
# probably should define those extensions to indicate media types:
#
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
#
# AddHandler allows you to map certain file extensions to "handlers":
# actions unrelated to filetype. These can be either built into the server
# or added with the Action directive (see below)
#
# To use CGI scripts outside of ScriptAliased directories:
# (You will also need to add "ExecCGI" to the "Options" directive.)
#
#AddHandler cgi-script .cgi
# For type maps (negotiated resources):
#AddHandler type-map var
#
# Filters allow you to process content before it is sent to the client.
#
# To parse .shtml files for server-side includes (SSI):
# (You will also need to add "Includes" to the "Options" directive.)
#
#AddType text/html .shtml
#AddOutputFilter INCLUDES .shtml
</IfModule>
#
# The mod_mime_magic module allows the server to use various hints from the
# contents of the file itself to determine its type. The MIMEMagicFile
# directive tells the module where the hint definitions are located.
#
#MIMEMagicFile conf/magic
#
# Customizable error responses come in three flavors:
# 1) plain text 2) local redirects 3) external redirects
#
# Some examples:
#ErrorDocument 500 "The server made a boo boo."
#ErrorDocument 404 /missing.html
#ErrorDocument 404 "/cgi-bin/missing_handler.pl"
#ErrorDocument 402 http://www.example.com/subscription_info.html
#
#
# MaxRanges: Maximum number of Ranges in a request before
# returning the entire resource, or one of the special
# values 'default', 'none' or 'unlimited'.
# Default setting is to accept 200 Ranges.
#MaxRanges unlimited
#
# EnableMMAP and EnableSendfile: On systems that support it,
# memory-mapping or the sendfile syscall may be used to deliver
# files. This usually improves server performance, but must
# be turned off when serving from networked-mounted
# filesystems or if support for these functions is otherwise
# broken on your system.
# Defaults: EnableMMAP On, EnableSendfile Off
#
#EnableMMAP off
#EnableSendfile on
# Supplemental configuration
#
# The configuration files in the conf/extra/ directory can be
# included to add extra features or to modify the default configuration of
# the server, or you may simply copy their contents here and change as
# necessary.
# Server-pool management (MPM specific)
#Include conf/extra/httpd-mpm.conf
# Multi-language error messages
#Include conf/extra/httpd-multilang-errordoc.conf
# Fancy directory listings
#Include conf/extra/httpd-autoindex.conf
# Language settings
#Include conf/extra/httpd-languages.conf
# User home directories
#Include conf/extra/httpd-userdir.conf
# Real-time info on requests and configuration
#Include conf/extra/httpd-info.conf
# Virtual hosts
#Include conf/extra/httpd-vhosts.conf
# Local access to the Apache HTTP Server Manual
#Include conf/extra/httpd-manual.conf
# Distributed authoring and versioning (WebDAV)
#Include conf/extra/httpd-dav.conf
# Various default settings
#Include conf/extra/httpd-default.conf
# Configure mod_proxy_html to understand HTML4/XHTML1
<IfModule proxy_html_module>
Include conf/extra/proxy-html.conf
</IfModule>
# Secure (SSL/TLS) connections
#Include conf/extra/httpd-ssl.conf
#
# Note: The following must must be present to support
# starting without SSL on platforms with no /dev/random equivalent
# but a statically compiled-in mod_ssl.
#
<IfModule ssl_module>
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
</IfModule>

60
apache/setup/index.html Normal file
View File

@ -0,0 +1,60 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<title>CKAN Docker Apache landing page</title>
<style>
body {
font-family: Arial, Helvetica, sans-serif;
text-align: center;
}
h2 {
font-weight: bold;
font-size:28px;
}
ul {
list-style-type: none;
margin: 0;
padding: 0;
}
li {
display: inline-block;
margin: 0 10px;
width: 200px;
}
li img {
display: block;
margin: 0 auto;
max-width: 200px;
}
li a {
display: block;
font-weight: bold;
text-decoration: none;
font-size:18px;
}
</style>
</head>
<body>
<h2>
CKAN Docker Apache landing page
</h2>
<ul>
<li>
<a href="/catalog">
<img src="https://upload.wikimedia.org/wikipedia/en/e/e2/CKAN_Logo_full_color.png" alt="Open Data Catalog">
</a>
<a href="/catalog">Open Data portal</a>
</li>
<li>
<a href="/csw">
<img src="https://www.ogc.org/wp-content/uploads/2023/03/OGC-1.svg" alt="CSW Endpoint">
</a>
<a href="/csw">CSW Endpoint</a>
</li>
</ul>
</body>
</html>

24
ckan-pycsw/Dockerfile Normal file
View File

@ -0,0 +1,24 @@
# Base image
FROM ghcr.io/mjanez/ckan-pycsw:main AS base
LABEL maintainer="mnl.janez@gmail.com"
# ckan-pycsw envvars
ENV APP_DIR=/app
ENV TZ=UTC
ENV PYCSW_CKAN_SCHEMA=iso19139_inspire
ENV PYCSW_CONFIG=${APP_DIR}/pycsw.conf
ENV CKAN_URL=http://localhost:5000/
ENV PYCSW_PORT=8000
ENV PYCSW_URL=http://localhost:${PYCSW_PORT}/
ENV DEV_MODE=False
ENV TIMEOUT=300
ENV PYCSW_CRON_DAYS_INTERVAL=2
ENV PYCSW_CRON_HOUR_START=4
ENV SSL_UNVERIFIED_MODE=False
WORKDIR ${APP_DIR}
# Update files if needed
COPY pycsw/conf/pycsw.conf.template docker-entrypoint.d/entrypoint.sh ./
ENTRYPOINT ["/bin/bash", "./entrypoint.sh"]

View File

@ -0,0 +1,30 @@
#!/bin/bash
set -xeuo pipefail
envsubst < pycsw.conf.template > pycsw.conf
# TODO: -Xfrozen_modules=off from: https://bugs.python.org/issue1666807
# Check if SSL_UNVERIFIED_MODE is enabled
if [ "${SSL_UNVERIFIED_MODE:-false}" = "true" ] || [ "${SSL_UNVERIFIED_MODE:-false}" = "True" ]; then
export REQUESTS_CA_BUNDLE=""
export CURL_CA_BUNDLE=""
SSL_FLAGS="--insecure" # Add SSL ignore flag
echo "[INSECURE] SSL_UNVERIFIED_MODE is enabled. SSL certificate verification is disabled."
else
SSL_FLAGS=""
fi
# Use curl directly instead of wait-for if necessary
echo 'Waiting for $CKAN_URL to become available...'
until curl $SSL_FLAGS --output /dev/null --silent --head --fail "$CKAN_URL"; do
printf '.'
sleep 30
done
echo 'CKAN is available.'
# Ejecutar el comando Python
pdm run python3 -Xfrozen_modules=off ckan2pycsw/ckan2pycsw.py
exec "$@"

View File

@ -0,0 +1,69 @@
[server]
home=${PWD}
url=${PYCSW_URL}
mimetype=application/xml; charset=UTF-8
encoding=UTF-8
language=en-US
maxrecords=10
#loglevel=DEBUG
#logfile=/tmp/pycsw.log
#ogc_schemas_base=http://foo
#federatedcatalogues=http://catalog.data.gov/csw
pretty_print=true
# Disable gzip_compresslevel when use httpd
#gzip_compresslevel=9
#domainquerytype=range
#domaincounts=true
#spatial_ranking=true
profiles=apiso
[manager]
transactions=false
allowed_ips=127.0.0.1
#csw_harvest_pagesize=10
[metadata:main]
identification_title=${CSW_IDENTIFICATION_TITLE}
identification_abstract=${CSW_IDENTIFICATION_ABSTRACT}
identification_keywords=catalogue,discovery,metadata,spatial
identification_keywords_type=theme
identification_fees=None
identification_accessconstraints=None
provider_name=${CSW_PROVIDER_NAME}
provider_url=${CSW_PROVIDER_URL}
contact_name=${CSW_CONTACT_NAME}
contact_position=${CSW_CONTACT_POSITION}
contact_address=${CSW_CONTACT_ADDRESS}
contact_city=${CSW_CONTACT_CITY}
contact_stateorprovince=${CSW_CONTACT_STATE_OR_PROVINCE}
contact_postalcode=${CSW_CONTACT_POSTAL_CODE}
contact_country=${CSW_CONTACT_COUNTRY}
#contact_phone=+xx-xxx-xxx-xxxx
#contact_fax=+xx-xxx-xxx-xxxx
contact_email=${CSW_CONTACT_EMAIL}
contact_url=${CSW_CONTACT_URL}
contact_hours=Hours of Service
contact_instructions=During hours of service. Off on weekends.
contact_role=pointOfContact
[repository]
# sqlite
database=sqlite:///${PWD}/cite.db
# postgres
#database=postgresql://ckan:ckan@db/pycsw
# mysql
#database=mysql://username:password@localhost/pycsw?charset=utf8
#mappings=path/to/mappings.py
table=records
#filter=type='http://purl.org/dc/dcmitype/Dataset'
[metadata:inspire]
enabled=true
languages_supported=eng,spa
default_language=eng
date=${CSW_INSPIRE_DATE}
gemet_keywords=${CSW_INSPIRE_GEMET_KEYWORDS}
conformity_service=${CSW_INSPIRE_CONFORMITY}
contact_name=${CSW_INSPIRE_CONTACT_NAME}
contact_email=${CSW_INSPIRE_CONTACT_EMAIL}
temp_extent=${CSW_INSPIRE_TEMP_EXTENT}

154
ckan/Dockerfile Normal file
View File

@ -0,0 +1,154 @@
FROM harbor.d4science.org/d4science-ckan/ckan-spatial-base:2.10.5-py3.10
LABEL maintainer="mnl.janez@gmail.com"
# Set up environment variables
ENV APP_DIR=/srv/app
# ENV TZ=UTC
ENV TZ=Europe/Rome
RUN apt-get update \
&& apt-get install -yq tzdata locales \
&& sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen \
&& echo 'LANG="en_US.UTF-8"' > /etc/default/locale \
&& locale-gen \
&& echo "${TZ}" > /etc/timezone \
&& dpkg-reconfigure tzdata
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
# Branch Theme argument
# e.g.: main
ARG CACHEBUST
ARG THEME_BRANCH
# Convert ARG to ENV so it's available in RUN commands
ENV THEME_BRANCH=${THEME_BRANCH}
# Set working directory
WORKDIR ${APP_DIR}
# requirements.txt files fixed until next releases
COPY req_fixes req_fixes
# CKAN configuration & extensions
## XLoader - 1.0.1-stable (github.com/mjanez/ckanext-xloader) ##
## Harvest - fix_validation (code-repo.d4science.org/D4ScienceCatalogue/harvester-d4science) ##
## GoogleAnalytics - master (github.com/ckan/ckanext-googleanalytics) ##
## KeyCloak - master (code-repo.d4science.org/D4ScienceCatalogue/ckanext-keycloak) ##
## Geoview - v0.2.2 (github.com/ckan/ckanext-geoview) ##
## Spatial - default branch (code-repo.d4science.org/D4ScienceCatalogue/spatial-d4science) ##
## DCAT - v2.3.0 (github.com/ckan/ckanext-dcat) ##
## Scheming - release-3.0.0 (github.com/ckan/ckanext-scheming) ##
## Resource dictionary - v1.0.2 (github.com/mjanez/ckanext-resourcedictionary) ##
## Pages - v0.5.2 (github.com/ckan/ckanext-pages) ##
## PDFView - 0.0.8 (github.com/ckan/ckanext-pdfview) ##
## Fluent - v1.0.1 (github.com/mjanez/ckanext-fluent) ##
## Scheming DCAT - v4.5.3 (github.com/mjanez/ckanext-schemingdcat) ##
## GeoNetwork - main (code-repo.d4science.org/D4ScienceCatalogue/ckanext-geonetwork) ##
## Hierarchy - v1.2.2 (github.com/ckan/ckanext-hierarchy) ##
## d4science Theme - main / custom ${THEME_BRANCH} (code-repo.d4science.org/D4ScienceCatalogue/ckanext-d4science_theme) ##
RUN echo "Cache bust: ${CACHEBUST}" && \
# echo ${TZ} > /etc/timezone && \
# if ! [ /usr/share/zoneinfo/${TZ} -ef /etc/localtime ]; then cp /usr/share/zoneinfo/${TZ} /etc/localtime; fi && \
# Install patch utility
apt-get update && \
apt-get install -y patch vim && \
# Install cron for scheduled tasks
apt-get install -y cron && \
# Install CKAN extensions
echo "ckan/ckanext-xloader" && \
pip3 install --no-cache-dir -e git+https://github.com/mjanez/ckanext-xloader.git@1.0.1-stable#egg=ckanext-xloader && \
pip3 install --no-cache-dir -r ${APP_DIR}/src/ckanext-xloader/requirements.txt && \
pip3 install --no-cache-dir -U requests[security] && \
echo "ckan/ckanext-harvest" && \
pip3 install --no-cache-dir -e git+https://code-repo.d4science.org/D4ScienceCatalogue/harvester-d4science.git@fix_validation#egg=ckanext-harvest && \
pip3 install --no-cache-dir -r ${APP_DIR}/src/ckanext-harvest/pip-requirements.txt && \
echo "ckan/ckanext-geoview" && \
pip3 install --no-cache-dir -e git+https://github.com/ckan/ckanext-geoview.git@v0.2.2#egg=ckanext-geoview && \
echo "ckan/ckanext-spatial" && \
pip3 install --no-cache-dir -e git+https://code-repo.d4science.org/D4ScienceCatalogue/spatial-d4science.git#egg=ckanext-spatial && \
pip3 install --no-cache-dir -r ${APP_DIR}/req_fixes/ckanext-spatial/requirements.txt && \
echo "mjanez/ckanext-dcat (GeoDCAT-AP extended version)" && \
pip3 install --no-cache-dir -e git+https://github.com/ckan/ckanext-dcat.git@v2.3.0#egg=ckanext-dcat && \
pip3 install --no-cache-dir -r ${APP_DIR}/src/ckanext-dcat/requirements.txt && \
echo "ckan/ckanext-scheming" && \
pip3 install --no-cache-dir -e git+https://github.com/ckan/ckanext-scheming.git@release-3.0.0#egg=ckanext-scheming && \
echo "mjanez/ckanext-resourcedictionary" && \
pip3 install --no-cache-dir -e git+https://github.com/mjanez/ckanext-resourcedictionary.git@v1.0.2#egg=ckanext-resourcedictionary && \
echo "ckan/ckanext-pages" && \
pip3 install --no-cache-dir -e git+https://github.com/ckan/ckanext-pages.git@v0.5.2#egg=ckanext-pages && \
echo "ckan/ckanext-pdfview" && \
pip3 install --no-cache-dir -e git+https://github.com/ckan/ckanext-pdfview.git@0.0.8#egg=ckanext-pdfview && \
echo "mjanez/ckanext-fluent" && \
pip3 install --no-cache-dir -e git+https://github.com/mjanez/ckanext-fluent.git@v1.0.1#egg=ckanext-fluent && \
# echo "mjanez/ckanext-schemingdcat" && \
# pip3 install --no-cache-dir -e git+https://github.com/mjanez/ckanext-schemingdcat.git@v4.5.3#egg=ckanext_schemingdcat && \
# pip3 install --no-cache-dir -r ${APP_DIR}/src/ckanext-schemingdcat/requirements.txt && \
echo "ckan/ckanext-googleanalytics" && \
pip3 install --no-cache-dir -e git+https://github.com/ckan/ckanext-googleanalytics.git@master#egg=ckanext-googleanalytics && \
pip3 install --no-cache-dir -r ${APP_DIR}/src/ckanext-googleanalytics/requirements.txt && \
echo "ckan/ckanext-keycloak" && \
#pip3 install --no-cache-dir -e git+https://github.com/keitaroinc/ckanext-keycloak.git#egg=ckanext-keycloak && \
pip3 install --no-cache-dir -e git+https://code-repo.d4science.org/D4ScienceCatalogue/ckanext-keycloak.git@master#egg=ckanext-keycloak && \
pip3 install --no-cache-dir -r ${APP_DIR}/src/ckanext-keycloak/requirements.txt && \
echo "ckan/ckanext-geonetwork" && \
pip3 install --no-cache-dir -e git+https://code-repo.d4science.org/D4ScienceCatalogue/ckanext-geonetwork.git@main#egg=ckanext-geonetwork && \
echo "ckan/ckanext-hierarchy" && \
pip3 install --no-cache-dir -e git+https://github.com/ckan/ckanext-hierarchy.git@v1.2.2#egg=ckanext-hierarchy && \
pip3 install --no-cache-dir -r ${APP_DIR}/src/ckanext-hierarchy/requirements.txt && \
echo "ckanext/ckanext-d4science_theme" && \
pip3 install --no-cache-dir -e git+https://code-repo.d4science.org/D4ScienceCatalogue/ckanext-d4science_theme.git@main#egg=ckanext-d4science_theme && \
git clone -b ${THEME_BRANCH} https://code-repo.d4science.org/D4ScienceCatalogue/ckanext-d4science_theme.git /tmp/ckanext-d4science_theme_custom && \
pip3 install --no-cache-dir -r ${APP_DIR}/src/ckanext-d4science-theme/requirements.txt && \
# #echo "NaturalHistoryMuseum/ckanext-doi" && \
# #pip3 install --no-cache-dir ckanext-doi && \
# # Remove system cache
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# Used to configure the container environment by setting environment variables, creating users, running initialization scripts, .etc
COPY docker-entrypoint.d/* /docker-entrypoint.d/
# Apply any patches needed to CKAN core
COPY patches patches
RUN for d in $APP_DIR/patches/*; do \
if [ -d $d ]; then \
for f in `ls $d/*.patch | sort -g`; do \
cd $SRC_DIR/`basename "$d"` && \
if patch -R --dry-run -p1 < "$f"; then \
echo "$0: Patch $f has already been applied or reversed, skipping..."; \
else \
echo "$0: Applying patch $f to $SRC_DIR/`basename $d`"; \
sed -i 's/\r$//' "$f" && \
patch -p1 < "$f" ; \
fi \
done ; \
fi ; \
done
# Fix ownership of /srv/app/src once during build (instead of at every container startup)
RUN chown -R ckan:ckan /srv/app/src && \
rm -rf /var/lib/apt/lists/*
# Workers
## Update prerun.py with custom version
COPY setup/prerun.py.override ${APP_DIR}/prerun.py
## Update start_ckan.sh with custom workers
COPY setup/start_ckan.sh.override ${APP_DIR}/start_ckan.sh
RUN chmod +x ${APP_DIR}/start_ckan.sh
## Load workers supervisor configuration
COPY setup/workers/* /etc/supervisord.d/
# Start CKAN
CMD ["/bin/sh", "-c", "$APP_DIR/start_ckan.sh"]

124
ckan/Dockerfile.dev Normal file
View File

@ -0,0 +1,124 @@
FROM ghcr.io/mjanez/ckan-spatial-dev:2.10.5-py3.10
LABEL maintainer="francesco.mangiacrapa@isti.cnr.it"
LABEL maintainer="marco.procaccini@igg.cnr.it"
# Set up environment variables
ENV APP_DIR=/srv/app \
TZ=UTC \
SRC_EXTENSIONS_DIR=/srv/app/src_extensions
# requirements.txt files fixed until next releases
COPY req_fixes req_fixes
# Set working directory
WORKDIR ${APP_DIR}
# Build argument for cache busting
ARG CACHEBUST
# Branch Theme argument
ARG THEME_BRANCH=main
ENV THEME_BRANCH=${THEME_BRANCH}
# CKAN configuration & extensions
## XLoader - 1.0.1-stable (mjanez/Forked fixed version) ##
## Harvest - v1.5.6 (Worker with supervisor) ##
## GoogleAnalytics ##
## KeyCloak ##
## Geoview - v0.2.2 ##
## Spatial - v2.1.1 ##
## DCAT - v2.3.0 (Latest stable version of ckanext-dcat with minor fixes) ##
## Scheming - release-3.0.0 ##
## Resource dictionary - v1.0.2 (mjanez/Fixed version) ##
## Pages - v0.5.2 ##
## PDFView - 0.0.8 ##
## Fluent - v1.0.1 (mjanez/Forked stable version) ##
## Scheming DCAT - v4.0.0 (mjanez/GeoDCAT-AP/NTI-RISP extended version) ##
RUN echo "Cache bust: ${CACHEBUST}" && \
# echo ${TZ} > /etc/timezone && \
# if ! [ /usr/share/zoneinfo/${TZ} -ef /etc/localtime ]; then cp /usr/share/zoneinfo/${TZ} /etc/localtime; fi && \
# Install patch utility
apt-get update && \
apt-get install -y patch vim && \
# Install cron for scheduled tasks
apt-get install -y cron && \
# Install CKAN extensions
echo "ckan/ckanext-xloader" && \
pip3 install --no-cache-dir -e git+https://github.com/mjanez/ckanext-xloader.git@1.0.1-stable#egg=ckanext-xloader && \
pip3 install --no-cache-dir -r ${APP_DIR}/src/ckanext-xloader/requirements.txt && \
pip3 install --no-cache-dir -U requests[security] && \
echo "ckan/ckanext-harvest" && \
pip3 install --no-cache-dir -e git+https://code-repo.d4science.org/D4ScienceCatalogue/harvester-d4science.git@fix_validation#egg=ckanext-harvest && \
pip3 install --no-cache-dir -r ${APP_DIR}/src/ckanext-harvest/pip-requirements.txt && \
echo "ckan/ckanext-geoview" && \
pip3 install --no-cache-dir -e git+https://github.com/ckan/ckanext-geoview.git@v0.2.2#egg=ckanext-geoview && \
echo "ckan/ckanext-spatial" && \
pip3 install --no-cache-dir -e git+https://code-repo.d4science.org/D4ScienceCatalogue/spatial-d4science.git#egg=ckanext-spatial && \
pip3 install --no-cache-dir -r ${APP_DIR}/req_fixes/ckanext-spatial/requirements.txt && \
echo "mjanez/ckanext-dcat (GeoDCAT-AP extended version)" && \
pip3 install --no-cache-dir -e git+https://github.com/ckan/ckanext-dcat.git@v2.3.0#egg=ckanext-dcat && \
pip3 install --no-cache-dir -r ${APP_DIR}/src/ckanext-dcat/requirements.txt && \
echo "ckan/ckanext-scheming" && \
pip3 install --no-cache-dir -e git+https://github.com/ckan/ckanext-scheming.git@release-3.0.0#egg=ckanext-scheming && \
echo "mjanez/ckanext-resourcedictionary" && \
pip3 install --no-cache-dir -e git+https://github.com/mjanez/ckanext-resourcedictionary.git@v1.0.2#egg=ckanext-resourcedictionary && \
echo "ckan/ckanext-pages" && \
pip3 install --no-cache-dir -e git+https://github.com/ckan/ckanext-pages.git@v0.5.2#egg=ckanext-pages && \
echo "ckan/ckanext-pdfview" && \
pip3 install --no-cache-dir -e git+https://github.com/ckan/ckanext-pdfview.git@0.0.8#egg=ckanext-pdfview && \
echo "mjanez/ckanext-fluent" && \
pip3 install --no-cache-dir -e git+https://github.com/mjanez/ckanext-fluent.git@v1.0.1#egg=ckanext-fluent && \
# echo "mjanez/ckanext-schemingdcat" && \
# pip3 install --no-cache-dir -e git+https://github.com/mjanez/ckanext-schemingdcat.git@v4.5.3#egg=ckanext_schemingdcat && \
# pip3 install --no-cache-dir -r ${APP_DIR}/src/ckanext-schemingdcat/requirements.txt && \
echo "ckan/ckanext-googleanalytics" && \
pip3 install --no-cache-dir -e git+https://github.com/ckan/ckanext-googleanalytics.git@master#egg=ckanext-googleanalytics && \
pip3 install --no-cache-dir -r ${APP_DIR}/src/ckanext-googleanalytics/requirements.txt && \
echo "ckan/ckanext-keycloak" && \
pip3 install --no-cache-dir -e git+https://code-repo.d4science.org/D4ScienceCatalogue/ckanext-keycloak.git@master#egg=ckanext-keycloak && \
pip3 install --no-cache-dir -r ${APP_DIR}/src/ckanext-keycloak/requirements.txt && \
echo "ckan/ckanext-geonetwork" && \
pip3 install --no-cache-dir -e git+https://code-repo.d4science.org/D4ScienceCatalogue/ckanext-geonetwork.git@main#egg=ckanext-geonetwork && \
echo "ckan/ckanext-hierarchy" && \
pip3 install --no-cache-dir -e git+https://code-repo.d4science.org/D4ScienceCatalogue/hierarchy-d4science.git@d4science_custom#egg=ckanext-hierarchy && \
pip3 install --no-cache-dir -r ${APP_DIR}/src/ckanext-hierarchy/requirements.txt && \
echo "ckanext/ckanext-d4science_theme" && \
pip3 install --no-cache-dir -e git+https://code-repo.d4science.org/D4ScienceCatalogue/ckanext-d4science_theme.git@main#egg=ckanext-d4science_theme && \
git clone -b ${THEME_BRANCH} https://code-repo.d4science.org/D4ScienceCatalogue/ckanext-d4science_theme.git /tmp/ckanext-d4science_theme_custom && \
pip3 install --no-cache-dir -r ${APP_DIR}/src/ckanext-d4science-theme/requirements.txt && \
# #echo "NaturalHistoryMuseum/ckanext-doi" && \
# #pip3 install --no-cache-dir ckanext-doi && \
# # Remove system cache
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# Used to configure the container environment by setting environment variables, creating users, running initialization scripts, .etc
COPY docker-entrypoint.d/* /docker-entrypoint.d/
# Apply any patches needed to CKAN core
COPY patches patches
RUN for d in $APP_DIR/patches/*; do \
if [ -d $d ]; then \
for f in `ls $d/*.patch | sort -g`; do \
cd $SRC_DIR/`basename "$d"` && \
if patch -R --dry-run -p1 < "$f"; then \
echo "$0: Patch $f has already been applied or reversed, skipping..."; \
else \
echo "$0: Applying patch $f to $SRC_DIR/`basename $d`"; \
sed -i 's/\r$//' "$f" && \
patch -p1 < "$f" ; \
fi \
done ; \
fi ; \
done
# Override prerun.py with custom version
COPY setup/prerun.py.override ${APP_DIR}/prerun.py
# Override start_ckan.sh with DEV sh
COPY setup/start_ckan_development.sh.override ./start_ckan_development.sh
RUN chmod +x ./start_ckan_development.sh
## Load workers supervisor configuration
COPY setup/workers/* /etc/supervisord.d/

48
ckan/Dockerfile.ghcr Normal file
View File

@ -0,0 +1,48 @@
FROM ghcr.io/mjanez/ckan-docker:ckan-2.10.5
LABEL maintainer="mnl.janez@gmail.com"
# Set up environment variables
ENV APP_DIR=/srv/app
ENV TZ=UTC
# Set working directory
WORKDIR ${APP_DIR}
# Make sure both files are not exactly the same
RUN echo ${TZ} > /etc/timezone && \
if ! [ /usr/share/zoneinfo/${TZ} -ef /etc/localtime ]; then \
cp /usr/share/zoneinfo/${TZ} /etc/localtime ; \
fi ;
# Used to configure the container environment by setting environment variables, creating users, running initialization scripts, .etc
COPY docker-entrypoint.d/* docker-entrypoint.d/
# Apply any patches needed to CKAN core
COPY patches patches
# Updated version of the Dockerfile RUN command that skips applying a patch if a reversed or previously applied patch is detected
RUN for d in $APP_DIR/patches/*; do \
if [ -d $d ]; then \
for f in `ls $d/*.patch | sort -g`; do \
cd $SRC_DIR/`basename "$d"` && \
if patch -R --dry-run -p1 < "$f"; then \
echo "$0: Patch $f has already been applied or reversed, skipping..."; \
else \
echo "$0: Applying patch $f to $SRC_DIR/`basename $d`"; \
sed -i 's/\r$//' "$f" && \
patch -p1 < "$f" ; \
fi \
done ; \
fi ; \
done
# Workers
## Update start_ckan.sh with custom workers
COPY setup/start_ckan.sh.override ${APP_DIR}/start_ckan.sh
RUN chmod +x ${APP_DIR}/start_ckan.sh
## Load workers supervisor configuration
COPY setup/workers/* /etc/supervisord.d/
# Start CKAN
CMD ["/bin/sh", "-c", "$APP_DIR/start_ckan.sh"]

View File

@ -0,0 +1,176 @@
#!/bin/bash
# This script is executed at container startup and is responsible for loading the settings defined in the envvars into the CKAN config file (ckan.ini) using the ckan config-tool command.
# ckanext-scheming/ckanext-schemingdcat: Update settings defined in the envvars (https://github.com/ckan/ckanext-pages#configuration)
# Clear index if CKAN_SKIP_REINDEX_ON_STARTUP is not set to true (default: true)
if [ "${CKAN_SKIP_REINDEX_ON_STARTUP:-true}" != "true" ]; then
echo "[docker-entrypoint.01_setup_ckanext_config] Clear index"
ckan -c $CKAN_INI search-index clear
else
echo "[docker-entrypoint.01_setup_ckanext_config] Skipping index clear (CKAN_SKIP_REINDEX_ON_STARTUP=true)"
fi
# ckan: Update core settings
echo "[docker-entrypoint.01_setup_ckanext_config] Loading CKAN core settings in the CKAN config file"
ckan config-tool $CKAN_INI \
"search.facets.default=$SEARCH__FACETS__DEFAULT" \
"ckan.datastore.sqlsearch.enabled=$CKAN__DATASTORE__SQLSEARCH__ENABLED"
# ckan: Enable or disable tracking
echo "[docker-entrypoint.01_setup_ckanext_config] Setting CKAN tracking enabled to: ${CKAN_TRACKING_ENABLED:-false}"
ckan config-tool $CKAN_INI \
"ckan.tracking_enabled=${CKAN_TRACKING_ENABLED:-false}" \
"ckan.site_title=${CKAN__SITE_TITLE:-CKAN Instance}"
# ckan: tracking information: Add tracking information settings to the CKAN config file
echo "[docker-entrypoint.01_setup_ckanext_config] Loading tracking information settings in the CKAN config file"
ckan config-tool $CKAN_INI \
"ckanext.d4science_theme.tracking_info_text=$CKANEXT__D4SCIENCE_THEME__TRACKING_INFO_TEXT" \
"ckanext.d4science_theme.tracking_start_date=$CKANEXT__D4SCIENCE_THEME__TRACKING_START_DATE"
# ckan: Install language configurations
echo "[docker-entrypoint.01_setup_ckanext_config] Setting CKAN locales: (default: $CKAN__LOCALE_DEFAULT), (order: $CKAN__LOCALE_ORDER), (offered: $CKAN__LOCALES_OFFERED)"
ckan config-tool $CKAN_INI \
"ckan.locale_default=${CKAN__LOCALE_DEFAULT}" \
"ckan.locale_order=${CKAN__LOCALE_ORDER}" \
"ckan.locales_offered=${CKAN__LOCALES_OFFERED}"
# ckan: Update log levels
echo "[docker-entrypoint.01_setup_ckanext_config] Setting CKAN log levels: (logger_root: $ROOT_LOG_LEVEL), (logger_ckan: $CKAN_LOG_LEVEL), (logger_ckanext: $CKANEXT_LOG_LEVEL)"
ckan config-tool $CKAN_INI -s logger_root "level=${ROOT_LOG_LEVEL:-WARNING}"
ckan config-tool $CKAN_INI -s logger_ckan "level=${CKAN_LOG_LEVEL:-WARNING}"
ckan config-tool $CKAN_INI -s logger_ckanext "level=${CKANEXT_LOG_LEVEL:-WARNING}"
# Add SMTP settings if CKAN__SMTP_ENABLED is true
if [ "$CKAN__SMTP_ENABLED" = "true" ]; then
echo "[docker-entrypoint.01_setup_ckanext_config] Adding SMTP settings to the CKAN config file"
ckan config-tool $CKAN_INI \
"smtp.server=$CKAN_SMTP_SERVER" \
"smtp.starttls=$CKAN_SMTP_STARTTLS" \
"smtp.user=$CKAN_SMTP_USER" \
"smtp.password=$CKAN_SMTP_PASSWORD" \
"smtp.mail_from=$CKAN_SMTP_MAIL_FROM" \
"smtp.reply_to=" \
"email_to=" \
"error_email_from="
fi
# ckanext-keycloak: Add Keycloak settings to the CKAN config file
echo "[docker-entrypoint.01_setup_ckanext_config] Loading ckanext-keycloak settings in the CKAN config file"
# Set default values for optional parameters
KEYCLOAK_BUTTON_STYLE=${CKANEXT__KEYCLOAK__BUTTON_STYLE:-""}
ckan config-tool $CKAN_INI \
"ckanext.keycloak.server_url=$CKANEXT__KEYCLOAK__SERVER_URL" \
"ckanext.keycloak.client_id=$CKANEXT__KEYCLOAK__CLIENT_ID" \
"ckanext.keycloak.realm_name=$CKANEXT__KEYCLOAK__REALM_NAME" \
"ckanext.keycloak.redirect_uri=$CKANEXT__KEYCLOAK__REDIRECT_URI" \
"ckanext.keycloak.client_secret_key=$CKANEXT__KEYCLOAK__CLIENT_SECRET_KEY" \
"ckanext.keycloak.button_style=$KEYCLOAK_BUTTON_STYLE" \
"ckanext.keycloak.enable_ckan_internal_login=$CKANEXT__KEYCLOAK__ENABLE_CKAN_INTERNAL_LOGIN"
# ckanext-schemingdcat: Update settings
echo "[docker-entrypoint.01_setup_ckanext_config] Loading ckanext-scheming and ckanext-schemingdcat settings into ckan.ini"
ckan config-tool $CKAN_INI \
"scheming.dataset_schemas=$CKANEXT__SCHEMINGDCAT_DATASET_SCHEMA" \
"scheming.group_schemas=$CKANEXT__SCHEMINGDCAT_GROUP_SCHEMAS" \
"scheming.organization_schemas=$CKANEXT__SCHEMINGDCAT_ORGANIZATION_SCHEMAS" \
"scheming.presets=$CKANEXT__SCHEMINGDCAT_PRESETS" \
"schemingdcat.facet_list=$CKANEXT__SCHEMINGDCAT_FACET_LIST" \
"schemingdcat.organization_custom_facets=$CKANEXT__SCHEMINGDCAT_ORGANIZATION_CUSTOM_FACETS" \
"schemingdcat.group_custom_facets=$CKANEXT__SCHEMINGDCAT_GROUP_CUSTOM_FACETS" \
"schemingdcat.geometadata_base_uri=$CKANEXT__SCHEMINGDCAT_GEOMETADATA_BASE_URI" \
"schemingdcat.default_package_item_icon=$CKANEXT__SCHEMINGDCAT_DEFAULT_PACKAGE_ITEM_ICON" \
"schemingdcat.default_package_item_show_spatial=$CKANEXT__SCHEMINGDCAT_DEFAULT_PACKAGE_ITEM_SHOW_SPATIAL" \
"schemingdcat.show_metadata_templates_toolbar=$CKANEXT__SCHEMINGDCAT_SHOW_METADATA_TEMPLATES_TOOLBAR" \
"schemingdcat.metadata_templates_search_identifier=$CKANEXT__METADATA_TEMPLATES_SEARCH_IDENTIFIER" \
"schemingdcat.endpoints_yaml=$CKANEXT__SCHEMINGDCAT_ENDPOINTS_YAML" \
"schemingdcat.social_github=$CKANEXT__SCHEMINGDCAT__SOCIAL_GITHUB" \
"schemingdcat.social_x=$CKANEXT__SCHEMINGDCAT__SOCIAL_X" \
"schemingdcat.social_linkedin=$CKANEXT__SCHEMINGDCAT__SOCIAL_LINKEDIN"
# ckanext-dcat: Add settings to the CKAN config file
echo "[docker-entrypoint.01_setup_ckanext_config] Loading ckanext-dcat settings in the CKAN config file"
ckan config-tool $CKAN_INI \
"ckanext.dcat.base_uri=$CKANEXT__DCAT__BASE_URI" \
"ckanext.dcat.catalog_endpoint=$CKANEXT__DCAT__DEFAULT_CATALOG_ENDPOINT" \
"ckanext.dcat.rdf.profiles=$CKANEXT__DCAT__RDF_PROFILES"
# ckan previews: Add CKAN Resource views to the CKAN config file
echo "[docker-entrypoint.01_setup_ckanext_config] Loading resource views in the CKAN config file"
ckan config-tool $CKAN_INI \
"ckan.views.default_views=$CKAN__VIEWS__DEFAULT_VIEWS" \
"ckan.preview.json_formats=$CKAN__PREVIEW__JSON_FORMATS" \
"ckan.preview.xml_formats=$CKAN__PREVIEW__XML_FORMATS" \
"ckan.preview.text_formats=$CKAN__PREVIEW__TEXT_FORMATS" \
"ckan.preview.loadable=$CKAN__PREVIEW__LOADABLE"
#FOCUSED DATASET TYPES <type>:<image_path/icon> images must be uploaded in /catalog/icons/tags-icons/
#CKAN__FOCUSED_DATASET_TYPES="Dataset:database.svg, Service:service_tag.svg, Deliverable:papers-text.svg, Guidelines:note-list-check.svg, Poster:tv.svg, Presentation:presentation-text.svg, Provider:speedometer.svg, Method:bookmark-star.svg, Lesson:book-text.svg"
#get icons from https://fontawesome.com
#CKAN__FOCUSED_DATASET_TYPES_ICONS="Dataset:fa-solid fa-database, Deliverable:fa-solid fa-newspaper, Guidelines:fa-solid fa-clipboard-list, Method:fa-solid fa-bookmark"
echo "[docker-entrypoint.01_setup_ckanext_config] Loading focused dataset types in the CKAN config file"
ckan config-tool $CKAN_INI \
"ckan.focused_dataset_types=$CKAN__FOCUSED_DATASET_TYPES" \
"ckan.focused_dataset_types_icons=$CKAN__FOCUSED_DATASET_TYPES_ICONS"
#ITINERIS customization: Add ckanext-custom-type settings to the CKAN config file
echo "[docker-entrypoint.01_setup_ckanext_config] Loading ckanext.custom.type settings in the CKAN config file"
ckan config-tool $CKAN_INI \
"ckanext.custom.type.titles=$CKANEXT__CUSTOM__TYPE__TITLES" \
"ckanext.custom.type.descriptions=$CKANEXT__CUSTOM__TYPE__DESCRIPTIONS" \
"ckanext.custom.type.facets=$CKANEXT__CUSTOM__TYPE__FACETS"
# ckanext-geoview: Add geoviews CKAN config file
echo "[docker-entrypoint.01_setup_ckanext_config] Loading geoviews in the CKAN config file"
ckan config-tool $CKAN_INI \
"ckanext.geoview.ol_viewer.formats=$CKANEXT__GEOVIEW__OL_VIEWER__FORMATS" \
"ckanext.geoview.shp_viewer.srid=$CKANEXT__GEOVIEW__SHP_VIEWER__SRID" \
"ckanext.geoview.shp_viewer.encoding=$CKANEXT__GEOVIEW__SHP_VIEWER__ENCODING" \
"ckanext.geoview.geojson.max_file_size=$CKANEXT__GEOVIEW__GEOJSON__MAX_FILE_SIZE"
# ckanext-spatial: Add spatial settings to CKAN config file
echo "[docker-entrypoint.01_setup_ckanext_config] Loading ckanext-spatial settings in the CKAN config file"
ckan config-tool $CKAN_INI \
"ckanext.spatial.search_backend=$CKANEXT__SPATIAL__SEARCH_BACKEND" \
"ckanext.spatial.harvest.continue_on_validation_errors=$CKANEXT__SPATIAL__HARVEST__CONTINUE_ON_VALIDATION_ERRORS" \
"ckan.spatial.srid=$CKAN__SPATIAL__SRID" \
"ckanext.spatial.common_map.type=$CKANEXT__SPATIAL__COMMON_MAP__TYPE" \
"ckanext.spatial.common_map.custom.url=$CKANEXT__SPATIAL__COMMON_MAP__CUSTOM__URL" \
"ckanext.spatial.common_map.attribution=$CKANEXT__SPATIAL__COMMON_MAP__ATTRIBUTION"
# ckanext-google_analytics: Add googleAnalytics CKAN config file
echo "[docker-entrypoint.01_setup_ckanext_config] Loading googleanalytics in the CKAN config file"
ckan config-tool $CKAN_INI \
"googleanalytics.id=$CKANEXT__GOOGLEANALYTICS_ID" \
# ckanext-pages: Add pages CKAN config file
echo "[docker-entrypoint.01_setup_ckanext_config] Loading pages config in the CKAN config file"
ckan config-tool $CKAN_INI \
"ckan.pages.allow_html=$CKANEXT__PAGES__ALOW_HTML" \
"ckanext.pages.organization=$CKANEXT__PAGES__ORGANIZATION" \
"ckanext.pages.group=$CKANEXT__PAGES__GROUP" \
"ckanext.pages.about_menu=$CKANEXT__PAGES__ABOUT_MENU" \
"ckanext.pages.group_menu=$CKANEXT__PAGES__GROUP_MENU" \
"ckanext.pages.organization_menu=$CKANEXT__PAGES__ORGANIZATION_MENU"
# ckanext-pages: Add pages CKAN config file
echo "[docker-entrypoint.01_setup_ckanext_config] Loading beaker settings in the CKAN config file"
ckan config-tool $CKAN_INI \
"beaker.session.samesite=${BEAKER_SESSION_SAMESITE:-None}" \
"beaker.session.secure=${BEAKER_SESSION_SECURE:-true}"
## TODO: log error "2025-03-11 15:26:40 ckan-dev-1 | Error: No such command 'doi'."
echo "[docker-entrypoint.01_setup_ckanext_config] Initialization DOI database"
ckan -c $CKAN_INI doi initdb
# Rebuild index if CKAN_SKIP_REINDEX_ON_STARTUP is not set to true (default: true)
if [ "${CKAN_SKIP_REINDEX_ON_STARTUP:-true}" != "true" ]; then
echo "[docker-entrypoint.01_setup_ckanext_config] Rebuild index"
ckan -c $CKAN_INI search-index rebuild
else
echo "[docker-entrypoint.01_setup_ckanext_config] Skipping index rebuild (CKAN_SKIP_REINDEX_ON_STARTUP=true)"
fi

View File

@ -0,0 +1,30 @@
#!/bin/bash
# Set the name of the token to revoke
TOKEN_NAME="xloader"
# Get the list of tokens and extract the IDs for tokens with the specified name
TOKEN_IDS=$(ckan -c $CKAN_INI user token list $CKAN_SYSADMIN_NAME | grep "$TOKEN_NAME" | awk -F'[][]' '{print $2}' | tr -d '[]')
# Revoke each previous token of xloader
for TOKEN_ID in $TOKEN_IDS
do
if [ -z "$TOKEN_ID" ]; then
echo "[docker-entrypoint.01_setup_xloader] No API Token to revoke"
continue
fi
ckan -c $CKAN_INI user token revoke -- $TOKEN_ID
if [ $? -eq 0 ]; then
echo "[docker-entrypoint.01_setup_xloader] API Token $TOKEN_ID has been revoked"
fi
done
# Add ckanext.xloader.api_token to the CKAN config file
echo "[docker-entrypoint.01_setup_xloader] Loading ckanext-xloader settings in the CKAN config file"
ckan config-tool $CKAN_INI \
"ckanext.xloader.api_token=xxx" \
"ckanext.xloader.jobs_db.uri=$CKANEXT__XLOADER__JOBS__DB_URI"
# Create ckanext-xloader API_TOKEN
echo "[docker-entrypoint.01_setup_xloader] Set up ckanext.xloader.api_token in the CKAN config file"
ckan config-tool $CKAN_INI "ckanext.xloader.api_token=$(ckan -c $CKAN_INI user token add $CKAN_SYSADMIN_NAME xloader | tail -n 1 | tr -d '\t')"

View File

@ -0,0 +1,104 @@
#!/bin/bash
echo "Starting D4Science CKAN customizations installation."
CKAN_APP_DIR=/srv/app/src
GCUBE_LANG_NAME=en
#GCUBE_LANG_NAME=en_gcube
GCUBE_SOURCE_LANG_DIR=${CKAN_APP_DIR}/ckanext-d4science-theme/languages/default
GCUBE_LANG_DEST_DIR=${CKAN_APP_DIR}/ckan/ckan/i18n/${GCUBE_LANG_NAME}/LC_MESSAGES
# install_gcube_lang - Installs and configures G-Cube language localization for CKAN
#
# This function sets up language localization by:
# 1. Creating the destination directory for language files if it doesn't exist
# 2. Compiling the language catalog for the specified locale
# Returns: No explicit return value
#
install_gcube_lang() {
# Install GCube language localization for CKAN
echo -e "\n*** Install_gcube_lang called"
if [ ! -d "${GCUBE_LANG_DEST_DIR}" ] ; then
if mkdir -p "${GCUBE_LANG_DEST_DIR}" 2>/dev/null ; then
echo "Created destination directory: ${GCUBE_LANG_DEST_DIR}"
fi
#GCUBE_INIT_LANG=0
fi
if [ ! -w "${GCUBE_LANG_DEST_DIR}" ] ; then
# chown may have failed at startup (rootless Docker / no CAP_CHOWN); try chmod as fallback
chmod ug+w "${GCUBE_LANG_DEST_DIR}" 2>/dev/null || true
fi
if [ ! -w "${GCUBE_LANG_DEST_DIR}" ] ; then
echo "${GCUBE_LANG_DEST_DIR} is not writable. Skipping language override/compile for locale ${GCUBE_LANG_NAME}."
echo -e "\n*** Install_gcube_lang finished with warnings: not writable destination directory."
return 0
fi
echo "Destination directory is writable: ${GCUBE_LANG_DEST_DIR}"
# Copy language files to destination
if [ -d "${GCUBE_SOURCE_LANG_DIR}" ] ; then
echo "Source language directory found: ${GCUBE_SOURCE_LANG_DIR}"
if cp -f "${GCUBE_SOURCE_LANG_DIR}/${GCUBE_LANG_NAME}/LC_MESSAGES/ckan.po" "${GCUBE_LANG_DEST_DIR}" ; then
echo "Copied ckan.po to ${GCUBE_LANG_DEST_DIR}"
#cp -f "${GCUBE_SOURCE_LANG_DIR}/i18n.py" "${GCUBE_I18N_DEST_DIR}/i18n.py"
else
echo "Unable to copy ckan.po to ${GCUBE_LANG_DEST_DIR}. Skipping language override for locale ${GCUBE_LANG_NAME}."
echo -e "\n*** Install_gcube_lang finished with warnings: unable to copy language files."
return 0
fi
else
echo "${GCUBE_SOURCE_LANG_DIR} not found. Skipping language override for locale ${GCUBE_LANG_NAME}."
echo -e "\n*** Install_gcube_lang finished with warnings: source directory not found."
return 0
fi
cd "${CKAN_APP_DIR}/ckan"
# Use --use-fuzzy flag to compile even if catalog is marked as fuzzy
if python setup.py compile_catalog --locale ${GCUBE_LANG_NAME} --use-fuzzy ; then
echo "Catalog compiled successfully for locale ${GCUBE_LANG_NAME}."
else
echo "Unable to compile catalog for locale ${GCUBE_LANG_NAME} due to filesystem permissions. Skipping."
echo -e "\n*** Install_gcube_lang finished with warnings: unable to compile catalog."
return 0
fi
# INSTALL ONLY DURING INIT
#if [ $GCUBE_INIT_LANG -eq 0 ] ; then
# . /usr/lib/ckan/default/bin/activate
# cd "${CKAN_APP_DIR}/ckan"
# python setup.py compile_catalog --locale ${GCUBE_LANG_NAME}
#fi
echo -e "\n*** Install_gcube_lang finished successfully"
}
# License installation
LICENSES_FILE_NAME=gcube-data-catalogue-licenses.json
LICENSES_SOURCE_FILE=${CKAN_APP_DIR}/ckanext-d4science-theme/licenses/${LICENSES_FILE_NAME}
install_gcube_licenses() {
############## INSTALL LICENSES ################
echo -e "\n*** Install_gcube_licenses called"
# Copy licenses file to CKAN config directory
if [ -f "${LICENSES_SOURCE_FILE}" ] ; then
ckan config-tool $CKAN_INI "licenses_group_url=${LICENSES_SOURCE_FILE}"
echo "Update licenses in ckan.ini from ${LICENSES_SOURCE_FILE}"
else
echo "${LICENSES_SOURCE_FILE} not found in source directory. Skipping licenses_group_url config in ckan.ini"
fi
echo -e "\n*** Install_gcube_licenses finished"
}
########## Main ##########
install_gcube_lang
install_gcube_licenses
# echo "Setting en_gcube as primary locale"
# ckan config-tool $CKAN_INI \
# "ckan.locale_default=en_gcube" \
# "ckan.locales_offered=en_gcube en es pt_BR ja it cs_CZ ca fr el sv sr sr@latin no sk fi ru de pl nl bg ko_KR hu sa sl lv" \
# "ckan.locale_order=en_gcube en es pt_BR ja it cs_CZ ca fr el sv sr sr@latin no sk fi ru de pl nl bg ko_KR hu sa sl lv"
echo "Finished D4Science CKAN customizations installation."

View File

@ -0,0 +1,39 @@
#!/bin/bash
echo "Starting D4Science CKAN theme customizations installation."
CKAN_APP_DIR=/srv/app/src
#GCUBE_LANG_NAME=en_gcube
GCUBE_SOURCE_THEME_DIR=/tmp/ckanext-d4science_theme_custom
GCUBE_DEST_THEME_DIR=${CKAN_APP_DIR}/ckanext-d4science-theme
# install_gcube_lang - Installs and configures G-Cube language localization for CKAN
#
# This function sets up language localization by:
# 1. Creating the destination directory for language files if it doesn't exist
# 2. Compiling the language catalog for the specified locale
# Returns: No explicit return value
#
install_gcube_theme() {
# Install GCube theme for CKAN
echo -e "\n*** Install_gcube_theme called"
if [ ! -d ${GCUBE_DEST_THEME_DIR} ] ; then
echo "Destination theme directory ${GCUBE_DEST_THEME_DIR} does not exist. Exiting."
exit 1 # Destination theme directory does not exist
#GCUBE_INIT_LANG=0
fi
# Copy theme files to destination
if [ -d "${GCUBE_SOURCE_THEME_DIR}" ] ; then
echo "Installing gCube theme from ${GCUBE_SOURCE_THEME_DIR} to ${GCUBE_DEST_THEME_DIR}"
cp -r "${GCUBE_SOURCE_THEME_DIR}"/* "${GCUBE_DEST_THEME_DIR}"
rm -rf "${GCUBE_SOURCE_THEME_DIR}"
echo -e "\n*** Install_gcube_theme finished"
fi
}
# Call the function to install G-Cube theme
install_gcube_theme
echo "Finished D4Science CKAN theme customizations installation."

View File

@ -0,0 +1,53 @@
#!/bin/bash
echo "[docker-entrypoint.99_create_sysadmin] Creating or verifying sysadmin user"
name="${CKAN_SYSADMIN_NAME}"
password="${CKAN_SYSADMIN_PASSWORD}"
email="${CKAN_SYSADMIN_EMAIL}"
if [ -z "$name" ] || [ -z "$password" ] || [ -z "$email" ]; then
echo "[docker-entrypoint.99_create_sysadmin] CKAN_SYSADMIN_NAME, PASSWORD, and EMAIL must all be set"
exit 0
fi
# Check if user exists
if ckan -c $CKAN_INI user show "$name" 2>&1 | grep -q "name=$name"; then
echo "[docker-entrypoint.99_create_sysadmin] User $name already exists"
else
echo "[docker-entrypoint.99_create_sysadmin] Creating user $name"
ckan -c $CKAN_INI user add "$name" \
password="$password" \
email="$email"
if [ $? -eq 0 ]; then
echo "[docker-entrypoint.99_create_sysadmin] User $name created successfully"
else
echo "[docker-entrypoint.99_create_sysadmin] ERROR: Failed to create user $name"
exit 1
fi
fi
# Check if user is already a sysadmin
if ckan -c $CKAN_INI user show "$name" 2>&1 | grep -q "sysadmin: True"; then
echo "[docker-entrypoint.99_create_sysadmin] User $name is already a sysadmin"
else
# Make user sysadmin (provide 'y' answer to confirmation prompt)
echo "[docker-entrypoint.99_create_sysadmin] Making user $name a sysadmin"
echo "y" | ckan -c $CKAN_INI sysadmin add "$name"
if [ $? -eq 0 ]; then
echo "[docker-entrypoint.99_create_sysadmin] User $name is now a sysadmin"
else
echo "[docker-entrypoint.99_create_sysadmin] ERROR: Failed to make user $name a sysadmin"
exit 1
fi
fi
# Final verification
echo "[docker-entrypoint.99_create_sysadmin] Final verification of user $name"
if ckan -c $CKAN_INI user show "$name" 2>&1 | grep -q "sysadmin: True"; then
echo "[docker-entrypoint.99_create_sysadmin] ✓ User $name verified as sysadmin"
else
echo "[docker-entrypoint.99_create_sysadmin] ✗ WARNING: User $name exists but is not a sysadmin!"
fi

View File

@ -0,0 +1,4 @@
Use scripts in this folder to run extra initialization steps in your custom CKAN images.
Any file with `.sh` or `.py` extension will be executed just after the main initialization
script (`prerun.py`) is executed and just before the web server and supervisor processes are
started.

View File

@ -0,0 +1 @@
test

View File

@ -0,0 +1,16 @@
diff --git a/ckan/public/base/javascript/plugins/jquery.slug-preview.js b/ckan/public/base/javascript/plugins/jquery.slug-preview.js
index 4a4f78fe2..8c03e816d 100644
--- a/ckan/public/base/javascript/plugins/jquery.slug-preview.js
+++ b/ckan/public/base/javascript/plugins/jquery.slug-preview.js
@@ -33,6 +33,11 @@
var value = preview.find('.slug-preview-value');
var required = $('<div>').append($('.control-required', element).clone()).html();
+ // Check if slugPreview already exists in the parent container
+ if (element.parent().find('.slug-preview').length > 0) {
+ return;
+ }
+
function setValue() {
var val = escape(field.val()) || options.placeholder;
value.text(val);

View File

@ -0,0 +1,18 @@
diff --git a/ckan/logic/action/delete.py b/ckan/logic/action/delete.py
--- a/ckan/logic/action/delete.py
+++ b/ckan/logic/action/delete.py
@@ -152,6 +152,14 @@ def dataset_purge(context: Context, data_dict: DataDict) -> ActionResult.DatasetPurge
model.PackageRelationship.object_package_id == pkg.id)).all():
r.purge()
+ # Purge package extras - fix for ForeignKeyViolation on package_extra table
+ # This must happen before pkg.purge() to avoid foreign key constraint errors
+ # Added by Francesco Mangiacrapa - Incident #30926
+ for extra in model.Session.query(model.PackageExtra).filter(
+ model.PackageExtra.package_id == pkg.id).all():
+ model.Session.delete(extra)
+ model.Session.flush()
+
pkg = model.Package.get(id)
assert pkg
pkg.purge()

View File

@ -0,0 +1,32 @@
diff --git a/ckanext/pages/plugin.py b/ckanext/pages/plugin.py
index fbd1be0..2882f97 100644
--- a/ckanext/pages/plugin.py
+++ b/ckanext/pages/plugin.py
@@ -8,6 +8,7 @@ from ckan.plugins import toolkit as tk
import ckan.plugins as p
from ckan.lib.helpers import build_nav_main as core_build_nav_main
+from ckan.lib.helpers import lang
from ckanext.pages import actions, db
from ckanext.pages import auth
@@ -25,6 +26,10 @@ def build_pages_nav_main(*args):
about_menu = tk.asbool(tk.config.get('ckanext.pages.about_menu', True))
group_menu = tk.asbool(tk.config.get('ckanext.pages.group_menu', True))
org_menu = tk.asbool(tk.config.get('ckanext.pages.organization_menu', True))
+
+ # get the root path from the config and use lang if needed
+ root_path = tk.config.get('ckan.root_path', '')
+ root_path = root_path.replace('/{{LANG}}', '') if lang() == tk.config.get('ckan.locale_default') else root_path.replace('/{{LANG}}', '/{}'.format(lang()))
new_args = []
for arg in args:
@@ -51,7 +56,7 @@ def build_pages_nav_main(*args):
type_ = 'blog' if page['page_type'] == 'blog' else 'pages'
name = quote(page['name'])
title = html_escape(page['title'])
- link = tk.h.literal(u'<a href="/{}/{}">{}</a>'.format(type_, name, title))
+ link = tk.h.literal(u'<a href="{}/{}/{}">{}</a>'.format(root_path, type_, name, title))
if page['name'] == page_name:
li = tk.literal('<li class="active">') + link + tk.literal('</li>')
else:

View File

@ -0,0 +1,30 @@
diff --git a/ckanext/pages/theme/templates_group/group/read_base.html b/ckanext/pages/theme/templates_group/group/read_base.html
index bbdae8f..6bf00fa 100644
--- a/ckanext/pages/theme/templates_group/group/read_base.html
+++ b/ckanext/pages/theme/templates_group/group/read_base.html
@@ -3,8 +3,8 @@
{% block content_primary_nav %}
{{ super() }}
{% if h.ckan_version().split('.')[1] | int >= 9 %}
- {{ h.build_nav_icon('pages.group_pages_index', _('Pages'), id=c.group_dict.name, icon='file') }}
+ {{ h.build_nav_icon('pages.group_pages_index', _('Pages'), id=group_dict.name, icon='file') }}
{% else %}
- {{ h.build_nav_icon('group_pages_index', _('Pages'), id=c.group_dict.name, icon='file') }}
+ {{ h.build_nav_icon('group_pages_index', _('Pages'), id=group_dict.name, icon='file') }}
{% endif %}
{% endblock %}
diff --git a/ckanext/pages/theme/templates_organization/organization/read_base.html b/ckanext/pages/theme/templates_organization/organization/read_base.html
index e5f3b47..eb2f39b 100644
--- a/ckanext/pages/theme/templates_organization/organization/read_base.html
+++ b/ckanext/pages/theme/templates_organization/organization/read_base.html
@@ -3,8 +3,8 @@
{% block content_primary_nav %}
{{ super() }}
{% if h.ckan_version().split('.')[1] | int >= 9 %}
- {{ h.build_nav_icon('pages.organization_pages_index', _('Pages'), id=c.group_dict.name, icon='file') }}
+ {{ h.build_nav_icon('pages.organization_pages_index', _('Pages'), id=group_dict.name, icon='file') }}
{% else %}
- {{ h.build_nav_icon('organization_pages_index', _('Pages'), id=c.group_dict.name, icon='file') }}
+ {{ h.build_nav_icon('organization_pages_index', _('Pages'), id=group_dict.name, icon='file') }}
{% endif %}
{% endblock %}

View File

@ -0,0 +1,15 @@
ckantoolkit
lxml>=2.3
argparse
pyparsing>=2.1.10
requests>=1.1.0
cython==0.29.36; python_version < '3.9'
pyproj==2.6.1; python_version < '3.9'
pyproj==3.6.1; python_version >= '3.9'
# Compatibility with NumPy 2.0: https://github.com/shapely/shapely/issues/1972
numpy>=1.26,<2
Shapely==2.0.4
OWSLib==0.28.1
geojson==3.0.1

View File

@ -0,0 +1,259 @@
import os
import sys
import subprocess
import psycopg2
try:
from urllib.request import urlopen
from urllib.error import URLError
except ImportError:
from urllib2 import urlopen
from urllib2 import URLError
import time
import re
import json
ckan_ini = os.environ.get("CKAN_INI", "/srv/app/ckan.ini")
RETRY = 5
def update_plugins():
plugins = os.environ.get("CKAN__PLUGINS", "")
print(("[prerun] Setting the following plugins in {}:".format(ckan_ini)))
print(plugins)
cmd = ["ckan", "config-tool", ckan_ini, "ckan.plugins = {}".format(plugins)]
subprocess.check_output(cmd, stderr=subprocess.STDOUT)
print("[prerun] Plugins set.")
def check_main_db_connection(retry=None):
print("[prerun] check_main_db_connection run")
conn_str = os.environ.get("CKAN_SQLALCHEMY_URL")
if not conn_str:
print("[prerun] CKAN_SQLALCHEMY_URL not defined, not checking db")
return
return check_db_connection(conn_str, retry)
def check_datastore_db_connection(retry=None):
print("[prerun] check_datastore_db_connection run")
conn_str = os.environ.get("CKAN_DATASTORE_WRITE_URL")
if not conn_str:
print("[prerun] CKAN_DATASTORE_WRITE_URL not defined, not checking db")
return
return check_db_connection(conn_str, retry)
def check_db_connection(conn_str, retry=None):
print("[prerun] check_db_connection run")
if retry is None:
retry = RETRY
elif retry == 0:
print("[prerun] Giving up after 5 tries...")
sys.exit(1)
try:
connection = psycopg2.connect(conn_str)
except psycopg2.Error as e:
print(str(e))
print("[prerun] Unable to connect to the database, waiting...")
time.sleep(10)
check_db_connection(conn_str, retry=retry - 1)
else:
connection.close()
def check_solr_connection(retry=None):
print("[prerun] check_solr_connection run")
if retry is None:
retry = RETRY
elif retry == 0:
print("[prerun] Giving up after 5 tries...")
sys.exit(1)
url = os.environ.get("CKAN_SOLR_URL", "")
search_url = '{url}/schema/name?wt=json'.format(url=url)
try:
connection = urlopen(search_url)
except URLError as e:
print(str(e))
print("[prerun] Unable to connect to solr, waiting...")
time.sleep(10)
check_solr_connection(retry=retry - 1)
else:
import re
conn_info = connection.read()
schema_name = json.loads(conn_info)
if 'ckan' in schema_name['name']:
print('[prerun] Succesfully connected to solr and CKAN schema loaded')
else:
print('[prerun] Succesfully connected to solr, but CKAN schema not found')
def init_db():
db_command = ["ckan", "-c", ckan_ini, "db", "upgrade"]
print("[prerun] Initializing or upgrading db - start")
try:
subprocess.check_output(db_command, stderr=subprocess.STDOUT)
print("[prerun] Initializing or upgrading db - end")
except subprocess.CalledProcessError as e:
if "OperationalError" in str(e.output):
print("[prerun] Database not ready, waiting a bit before exit...")
time.sleep(5)
sys.exit(1)
else:
print(str(e))
print(e.output)
raise e
def init_datastore_db():
conn_str = os.environ.get("CKAN_DATASTORE_WRITE_URL")
if not conn_str:
print("[prerun] Skipping datastore initialization")
return
datastore_perms_command = ["ckan", "-c", ckan_ini, "datastore", "set-permissions"]
connection = psycopg2.connect(conn_str)
cursor = connection.cursor()
print("[prerun] Initializing datastore db - start")
try:
datastore_perms = subprocess.Popen(
datastore_perms_command, stdout=subprocess.PIPE
)
perms_sql = datastore_perms.stdout.read()
# Remove internal pg command as psycopg2 does not like it
perms_sql = re.sub(b'\\\\connect "(.*)"', b"", perms_sql)
# Skip if SQL is empty or whitespace only
if not perms_sql or not perms_sql.strip():
print("[prerun] No datastore permissions to set (empty SQL), skipping")
return
cursor.execute(perms_sql)
for notice in connection.notices:
print(notice)
connection.commit()
print("[prerun] Initializing datastore db - end")
print(datastore_perms.stdout.read())
except psycopg2.Error as e:
print("[prerun] Could not initialize datastore")
print(str(e))
except subprocess.CalledProcessError as e:
if "OperationalError" in e.output:
print(e.output)
print("[prerun] Database not ready, waiting a bit before exit...")
time.sleep(5)
sys.exit(1)
else:
print(e.output)
raise e
finally:
cursor.close()
connection.close()
def create_sysadmin():
print("[prerun] create_sysadmin run")
name = os.environ.get("CKAN_SYSADMIN_NAME")
password = os.environ.get("CKAN_SYSADMIN_PASSWORD")
email = os.environ.get("CKAN_SYSADMIN_EMAIL")
print("[prerun] creating sysadmin name: {}".format(name))
if name and password and email:
# Check if user exists
command = ["ckan", "-c", ckan_ini, "user", "show", name]
try:
out = subprocess.check_output(command, stderr=subprocess.STDOUT)
if b"User:None" not in re.sub(b"\s", b"", out):
print("[prerun] Sysadmin user {} exists, skipping creation".format(name))
return
except subprocess.CalledProcessError as e:
# User doesn't exist, we'll create it
print("[prerun] User does not exist, will create it")
# Create user
command = [
"ckan",
"-c",
ckan_ini,
"user",
"add",
name,
"password=" + password,
"email=" + email,
]
result = subprocess.call(command)
if result == 0:
print("[prerun] Created user {0}".format(name))
else:
print("[prerun] ERROR: Failed to create user {0} (exit code: {1})".format(name, result))
return
# Make it sysadmin
command = ["ckan", "-c", ckan_ini, "sysadmin", "add", name]
result = subprocess.call(command)
if result == 0:
print("[prerun] Made user {0} a sysadmin".format(name))
else:
print("[prerun] ERROR: Failed to make user {0} a sysadmin (exit code: {1})".format(name, result))
return
# Verify user was created successfully
verify_command = ["ckan", "-c", ckan_ini, "user", "show", name]
try:
verify_output = subprocess.check_output(verify_command, stderr=subprocess.STDOUT)
if name.encode() in verify_output:
print("[prerun] User {0} verified in database".format(name))
else:
print("[prerun] WARNING: User {0} not found in database after creation!".format(name))
except subprocess.CalledProcessError as e:
print("[prerun] WARNING: Could not verify user {0} in database: {1}".format(name, e))
# cleanup permissions
# We're running as root before pivoting to uwsgi and dropping privs
data_dir = "%s/storage" % os.environ.get('CKAN_STORAGE_PATH', '/var/lib/ckan')
command = ["chown", "-R", "ckan:ckan", data_dir]
subprocess.call(command)
print("[prerun] Ensured storage directory is owned by ckan")
else:
print("[prerun] CKAN_SYSADMIN_NAME, PASSWORD, and EMAIL must all be set")
if __name__ == "__main__":
maintenance = os.environ.get("MAINTENANCE_MODE", "").lower() == "true"
if maintenance:
print("[prerun] Maintenance mode, skipping setup...")
else:
check_main_db_connection()
init_db()
update_plugins()
check_datastore_db_connection()
init_datastore_db()
check_solr_connection()
# Sysadmin user creation moved to docker-entrypoint.d/99_create_sysadmin.sh
# to ensure it runs after all other initialization scripts
print("[prerun] Sysadmin creation delegated to entrypoint script 99_create_sysadmin.sh")

View File

@ -0,0 +1,140 @@
#!/bin/bash
# Set up the Secret key used by Beaker and Flask
# This can be overriden using a CKAN___BEAKER__SESSION__SECRET env var
if grep -qE "beaker.session.secret ?= ?$" ckan.ini
then
echo "Setting beaker.session.secret in ini file"
ckan config-tool $CKAN_INI "beaker.session.secret=$(python3 -c 'import secrets; print(secrets.token_urlsafe())')"
ckan config-tool $CKAN_INI "WTF_CSRF_SECRET_KEY=$(python3 -c 'import secrets; print(secrets.token_urlsafe())')"
JWT_SECRET=$(python3 -c 'import secrets; print("string:" + secrets.token_urlsafe())')
ckan config-tool $CKAN_INI "api_token.jwt.encode.secret=${JWT_SECRET}"
ckan config-tool $CKAN_INI "api_token.jwt.decode.secret=${JWT_SECRET}"
fi
# Github Codespaces workaround. Check if WTF_CSRF_ENABLED is explicitly set to False and update CKAN_INI
if [ "$WTF_CSRF_ENABLED" = "False" ]; then
echo "WTF_CSRF_ENABLED is set to False. Updating CKAN_INI."
ckan config-tool $CKAN_INI "WTF_CSRF_ENABLED=False"
fi
# Comment out smtp.* settings if CKAN__SMTP_ENABLED is not True
if [ "$CKAN__SMTP_ENABLED" != "True" ]; then
echo "Commenting out smtp.* settings in ini file"
sed -i 's/^\(smtp\..*\)$/#\1/' $CKAN_INI
fi
# Set googleanalytics.id before db upgrade to prevent plugin initialization errors
# This will be overridden later in docker-entrypoint.d scripts if CKANEXT__GOOGLEANALYTICS_ID is set
echo "Setting default googleanalytics.id in ini file"
if [ ! -z "$CKANEXT__GOOGLEANALYTICS_ID" ]; then
ckan config-tool $CKAN_INI "googleanalytics.id=$CKANEXT__GOOGLEANALYTICS_ID"
else
ckan config-tool $CKAN_INI "googleanalytics.id="
fi
# authorization settings, see #30050
###AUTHORIZATION SETTINGS
# Added by Francesco Mangiacrapa
echo "Loading authorization settings in the CKAN config file"
ckan config-tool $CKAN_INI \
"ckan.auth.anon_create_dataset=${CKAN__AUTH__ANON_CREATE_DATASET:-false}" \
"ckan.auth.create_unowned_dataset=${CKAN__AUTH__CREATE_UNOWNED_DATASET:-false}" \
"ckan.auth.create_dataset_if_not_in_organization=${CKAN__AUTH__CREATE_DATASET_IF_NOT_IN_ORGANIZATION:-false}" \
"ckan.auth.user_create_groups=${CKAN__AUTH__USER_CREATE_GROUPS:-false}" \
"ckan.auth.user_create_organizations=${CKAN__AUTH__USER_CREATE_ORGANIZATIONS:-false}" \
"ckan.auth.user_delete_groups=${CKAN__AUTH__USER_DELETE_GROUPS:-false}" \
"ckan.auth.user_delete_organizations=${CKAN__AUTH__USER_DELETE_ORGANIZATIONS:-false}" \
"ckan.auth.create_user_via_api=${CKAN__AUTH__CREATE_USER_VIA_API:-false}" \
"ckan.auth.create_user_via_web=${CKAN__AUTH__CREATE_USER_VIA_WEB:-false}" \
"ckan.auth.roles_that_cascade_to_sub_groups=${CKAN__AUTH__ROLES_THAT_CASCADE_TO_SUB_GROUPS:-admin}" \
"ckan.auth.public_user_details=${CKAN__AUTH__PUBLIC_USER_DETAILS:-true}" \
"ckan.auth.public_activity_stream_detail=${CKAN__AUTH__PUBLIC_ACTIVITY_STREAM_DETAIL:-false}" \
"ckan.auth.allow_dataset_collaborators=${CKAN__AUTH__ALLOW_DATASET_COLLABORATORS:-false}" \
"ckan.auth.allow_admin_collaborators=${CKAN__AUTH__ALLOW_ADMIN_COLLABORATORS:-false}" \
"ckan.auth.allow_collaborators_to_change_owner_org=${CKAN__AUTH__ALLOW_COLLABORATORS_TO_CHANGE_OWNER_ORG:-false}" \
"ckan.auth.create_default_api_keys=${CKAN__AUTH__CREATE_DEFAULT_API_KEYS:-true}" \
"ckan.auth.route_after_login=${CKAN__AUTH__ROUTE_AFTER_LOGIN:-dashboard.datasets}"
# Run the prerun script to init CKAN and create the default admin user
echo "Running prerun.py to initialize CKAN and create the default admin user"
python3 prerun.py
# Fix CKAN storage permissions for uploads (groups, organizations, etc.)
echo "Fixing CKAN storage permissions..."
mkdir -p /var/lib/ckan/storage/uploads
chown -R ckan:ckan /var/lib/ckan/storage
# Ensure proper ownership of the CKAN i18n folders (for mounted volumes only)
echo "Fixing i18n translations permissions..."
[ -d "/srv/app/src/ckan/ckan/i18n/" ] && chown -R ckan:ckan /srv/app/src/ckan/ckan/i18n/
[ -d "/srv/app/src/ckan/ckan/public/base/i18n/" ] && chown -R ckan:ckan /srv/app/src/ckan/ckan/public/base/i18n/
# Run any startup scripts provided by images extending this one
if [[ -d "/docker-entrypoint.d" ]]
then
for f in /docker-entrypoint.d/*; do
case "$f" in
*.sh) echo "$0: Running init file $f"; . "$f" ;;
*.py) echo "$0: Running init file $f"; python3 "$f"; echo ;;
*) echo "$0: Ignoring $f (not an sh or py file)" ;;
esac
echo
done
fi
# Create Harvester logs directory and change its ownership
mkdir -p $CKAN_LOGS_PATH/harvester
chown -R ckan:ckan $CKAN_LOGS_PATH/harvester
# Create xloader logs directory and change its ownership
mkdir -p $CKAN_LOGS_PATH/xloader
chown -R ckan:ckan $CKAN_LOGS_PATH/xloader
# Create ckan logs directory and change its ownership
mkdir -p $CKAN_LOGS_PATH/ckan
chown -R ckan:ckan $CKAN_LOGS_PATH/ckan
# Start CKAN only if prerun.py executed successfully
echo "[prerun] Checking if prerun.py executed successfully..."
if [ $? -eq 0 ]
then
# Start supervisord
echo "[prerun.workers] Loading the CKAN workers with supervisord..."
supervisord --configuration /etc/supervisord.conf &
# Workers
## Add the harvester background processes to crontab if they do not already exist
echo "[prerun.workers] Add harvester background procceses to crontab"
if ! crontab -l | grep -q "/usr/bin/supervisorctl start ckan_harvester_run"; then
{ crontab -l; echo "*/15 * * * * /usr/bin/supervisorctl start ckan_harvester_run"; } | crontab -
fi
## Clean-up mechanism for the harvest log table. 'ckan.harvest.log_timeframe'. The default time frame is 30 days
if ! crontab -l | grep -q "/usr/bin/supervisorctl start ckan_harvester_clean_log"; then
{ crontab -l; echo "0 5 */30 * * /usr/bin/supervisorctl start ckan_harvester_clean_log"; } | crontab -
fi
## Add SMTP notification process to crontab if CKAN__SMTP_ENABLED is True
if [ "$CKAN__SMTP_ENABLED" = "True" ]; then
echo "[prerun.workers] Add SMTP notification process to crontab"
if ! crontab -l | grep -q "ckan -c $CKAN_INI notify send_emails"; then
{ crontab -l; echo "@hourly echo '{}' | ckan -c $CKAN_INI notify send_emails > /dev/null"; } | crontab -
fi
fi
## Execute cron in the background
echo "[prerun.workers] Execute cron in the background"
cron -f &
# Wait until log file exists
while [ ! -f /var/log/ckan/ckan.log ]; do
sleep 1
done
# Tail CKAN logs
tail -f /var/log/ckan/ckan.log
else
echo "[prerun] failed...not starting CKAN."
fi

View File

@ -0,0 +1,151 @@
#!/bin/bash
# Install any local extensions in the src_extensions volume
echo "Looking for local extensions to install..."
echo "Extension dir contents:"
ls -la $SRC_EXTENSIONS_DIR
for i in $SRC_EXTENSIONS_DIR/*
do
if [ -d $i ];
then
if [ -f $i/pip-requirements.txt ];
then
pip install -r $i/pip-requirements.txt
echo "Found requirements file in $i"
fi
if [ -f $i/requirements.txt ];
then
pip install -r $i/requirements.txt
echo "Found requirements file in $i"
fi
if [ -f $i/dev-requirements.txt ];
then
pip install -r $i/dev-requirements.txt
echo "Found dev-requirements file in $i"
fi
if [ -f $i/setup.py ];
then
cd $i
python3 $i/setup.py develop
echo "Found setup.py file in $i"
cd $APP_DIR
fi
if [ -f $i/pyproject.toml ];
then
cd $i
pip install -e .
echo "Found pyproject.toml file in $i"
cd $APP_DIR
fi
# Point `use` in test.ini to location of `test-core.ini`
if [ -f $i/test.ini ];
then
echo "Updating \`test.ini\` reference to \`test-core.ini\` for plugin $i"
ckan config-tool $i/test.ini "use = config:../../src/ckan/test-core.ini"
fi
fi
done
# Set debug to true
echo "Enabling debug mode"
ckan config-tool $CKAN_INI -s DEFAULT "debug = true"
# Set up the Secret key used by Beaker and Flask
# This can be overriden using a CKAN___BEAKER__SESSION__SECRET env var
if grep -qE "beaker.session.secret ?= ?$" ckan.ini
then
echo "Setting beaker.session.secret in ini file"
ckan config-tool $CKAN_INI "beaker.session.secret=$(python3 -c 'import secrets; print(secrets.token_urlsafe())')"
ckan config-tool $CKAN_INI "WTF_CSRF_SECRET_KEY=$(python3 -c 'import secrets; print(secrets.token_urlsafe())')"
JWT_SECRET=$(python3 -c 'import secrets; print("string:" + secrets.token_urlsafe())')
ckan config-tool $CKAN_INI "api_token.jwt.encode.secret=${JWT_SECRET}"
ckan config-tool $CKAN_INI "api_token.jwt.decode.secret=${JWT_SECRET}"
fi
# Update the plugins setting in the ini file with the values defined in the env var
echo "Loading the following plugins: $CKAN__PLUGINS"
ckan config-tool $CKAN_INI "ckan.plugins = $CKAN__PLUGINS"
# Update test-core.ini DB, SOLR & Redis settings
echo "Loading test settings into test-core.ini"
ckan config-tool $SRC_DIR/ckan/test-core.ini \
"sqlalchemy.url = $TEST_CKAN_SQLALCHEMY_URL" \
"ckan.datastore.write_url = $TEST_CKAN_DATASTORE_WRITE_URL" \
"ckan.datastore.read_url = $TEST_CKAN_DATASTORE_READ_URL" \
"solr_url = $TEST_CKAN_SOLR_URL" \
"ckan.redis.url = $TEST_CKAN_REDIS_URL"
# Update local (file://) licenses for Development mode
if [ -n "$CKAN___LICENSES_GROUP_URL" ]; then
ckan config-tool $CKAN_INI "licenses_group_url=${CKAN___LICENSES_GROUP_URL}"
echo "Update licenses in CKAN"
fi
# Run the prerun script to init CKAN and create the default admin user
python3 prerun.py
# Fix i18n translations permissions for mounted volumes
echo "Fixing i18n translations permissions..."
chown -R ckan:ckan /srv/app/src/ckan/ckan/i18n/
chown -R ckan:ckan /srv/app/src/ckan/ckan/public/base/i18n/
# Fix CKAN storage permissions for uploads (groups, organizations, etc.)
echo "Fixing CKAN storage permissions..."
mkdir -p /var/lib/ckan/storage/uploads
chown -R ckan:ckan /var/lib/ckan/storage
# Run any startup scripts provided by images extending this one
if [[ -d "/docker-entrypoint.d" ]]
then
for f in /docker-entrypoint.d/*; do
case "$f" in
*.sh) echo "$0: Running init file $f"; . "$f" ;;
*.py) echo "$0: Running init file $f"; python3 "$f"; echo ;;
*) echo "$0: Ignoring $f (not an sh or py file)" ;;
esac
echo
done
fi
CKAN_RUN="ckan -c $CKAN_INI run -H 0.0.0.0"
CKAN_OPTIONS=""
# if [ "$USE_DEBUGPY_FOR_DEV" = true ] ; then
# pip install debugpy
# CKAN_RUN="python3 -m debugpy --listen 0.0.0.0:5678 $CKAN_RUN"
# CKAN_OPTIONS="$CKAN_OPTIONS --disable-reloader"
# fi
# if [ "$USE_HTTPS_FOR_DEV" = true ] ; then
# CKAN_OPTIONS="$CKAN_OPTIONS -C unsafe.cert -K unsafe.key"
# fi
# Create Harvester logs directory and change its ownership
mkdir -p $CKAN_LOGS_PATH/harvester
chown -R ckan:ckan $CKAN_LOGS_PATH/harvester
# Create xloader logs directory and change its ownership
mkdir -p $CKAN_LOGS_PATH/xloader
chown -R ckan:ckan $CKAN_LOGS_PATH/xloader
# Start supervisord
supervisord --configuration /etc/supervisord.conf &
# rif https://github.com/ckan/ckanext-harvest?tab=readme-ov-file#harvester-run
su ckan -c "ckan -c $CKAN_INI harvester fetch-consumer &"
su ckan -c "ckan -c $CKAN_INI harvester gather-consumer &"
su ckan -c "ckan -c $CKAN_INI harvester run &"
# Start the development server as the ckan user with automatic reload
while true; do
su ckan -c "$CKAN_RUN $CKAN_OPTIONS"
echo Exit with status $?. Restarting.
sleep 2
done
# Workers
# To start the Harvester worker
# ckan harvester run
# Clean-up mechanism for the harvest log table
ckan harvester clean-harvest-log

View File

@ -0,0 +1,39 @@
[program:ckan]
directory=%(ENV_APP_DIR)s
# Set the common uwsgi options.
## Add thunder-lock to prevent multiple workers from running the same job and buffer-size to prevent large headers
## To increase performance, you can adjust the number of processes (-p) (More info: https://uwsgi-docs.readthedocs.io/en/latest/Options.html & https://www.bloomberg.com/company/stories/configuring-uwsgi-production-deployment/)
command=uwsgi --thunder-lock
--ignore-sigpipe
--socket /tmp/uwsgi.sock
--wsgi-file /srv/app/wsgi.py
--module wsgi:application
--uid 92
--gid 92
--http [::]:5000
--master
--enable-threads
--lazy-apps
--vacuum
--harakiri $UWSGI_HARAKIRI
--max-requests 500
--max-worker-lifetime 3600
--reload-on-rss 1024
--processes 2
--threads 2
--buffer-size 32768
--disable-logging
--log-4xx
--log-5xx
--auto-procname
--procname-prefix 'ckan-'
user=ckan
numprocs=1
stdout_logfile=/var/log/ckan/ckan.log
stdout_logfile_maxbytes=50MB
stderr_logfile=/var/log/ckan/ckan.log
stderr_logfile_maxbytes=50MB
autostart=true
autorestart=true
startsecs=10
priority=1

View File

@ -0,0 +1,51 @@
[program:ckan_gather_consumer]
command=ckan harvester gather-consumer
user=ckan
numprocs=1
stdout_logfile=/var/log/harvester/gather_consumer.log
stdout_logfile_maxbytes=50MB
stderr_logfile=/var/log/harvester/gather_consumer.log
stderr_logfile_maxbytes=50MB
autostart=true
autorestart=true
startsecs=10
priority=1
[program:ckan_fetch_consumer]
command=ckan harvester fetch-consumer
user=ckan
numprocs=1
stdout_logfile=/var/log/harvester/fetch_consumer.log
stdout_logfile_maxbytes=50MB
stderr_logfile=/var/log/harvester/fetch_consumer.log
stderr_logfile_maxbytes=50MB
autostart=true
autorestart=true
startsecs=10
priority=2
[program:ckan_harvester_run]
command=ckan harvester run
user=ckan
numprocs=1
stdout_logfile=/var/log/harvester/ckan_harvester.log
stdout_logfile_maxbytes=25MB
stderr_logfile=/var/log/harvester/ckan_harvester.log
stderr_logfile_maxbytes=25MB
autostart=true
autorestart=false
startsecs=2
priority=3
[program:ckan_harvester_clean_log]
command=ckan harvester clean-harvest-log
user=ckan
numprocs=1
stdout_logfile=/var/log/harvester/ckan_harvester_clean_log.log
stdout_logfile_maxbytes=25MB
stderr_logfile=/var/log/harvester/ckan_harvester_clean_log.log
stderr_logfile_maxbytes=25MB
autostart=false
autorestart=false
startsecs=2
priority=4

View File

@ -0,0 +1,12 @@
[program:ckan_xloader]
command=ckan jobs worker default
user=ckan
numprocs=1
stdout_logfile=/var/log/xloader/ckan_xloader.log
stdout_logfile_maxbytes=100MB
stderr_logfile=/var/log/xloader/ckan_xloader.log
stderr_logfile_maxbytes=100MB
autostart=true
autorestart=true
startsecs=4
priority=1

View File

@ -0,0 +1,3 @@
# encoding: utf-8
# Need some content here to avoid the packaging stripping it out

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

5148
ckan_translations/ckan.pot Normal file

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Binary file not shown.

File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More