From 53d2d65cb10d22af216a6e132aa552090dc08d21 Mon Sep 17 00:00:00 2001 From: mnjnz <96422458+mjanez@users.noreply.github.com> Date: Mon, 27 Mar 2023 12:26:51 +0200 Subject: [PATCH] Add custom images of base Dockerfiles Now use ghcr.io/mjanez/ckan-base-spatial:master --- README.md | 19 +++--- ckan/Dockerfile | 110 +------------------------------ ckan/Dockerfile.dev | 9 +-- ckan/Dockerfile.spatial-vanilla | 112 -------------------------------- 4 files changed, 15 insertions(+), 235 deletions(-) delete mode 100644 ckan/Dockerfile.spatial-vanilla diff --git a/README.md b/README.md index 62ff06a..c9c3e61 100644 --- a/README.md +++ b/README.md @@ -24,11 +24,11 @@ Contains Docker images for the different components of CKAN Cloud and a Docker c >This is a **custom installation of Docker Compose** with specific extensions for spatial data and [GeoDCAT-AP](https://github.com/SEMICeu/GeoDCAT-AP)/[INSPIRE](https://github.com/INSPIRE-MIF/technical-guidelines) metadata [profiles](https://en.wikipedia.org/wiki/Geospatial_metadata). For official installations, please have a look: [CKAN documentation: Installation](https://docs.ckan.org/en/latest/maintaining/installing/index.html). Available components: -* CKAN custom image based on the official CKAN repo [ckan-docker-base](https://github.com/ckan/ckan-docker-base) +* CKAN custom image based on the official CKAN repo [ckan-docker-spatial](https://github.com/mjanez/ckan-docker-spatial)[^1] The non-CKAN images are as follows: * PostgreSQL: Official PostgreSQL image. Database files are stored in a named volume. -* Solr: CKAN's [pre-configured Solr image](https://github.com/ckan/ckan-solr). The index data is stored in a named volume and has a spatial schema. [^1] +* Solr: CKAN's [pre-configured Solr image](https://github.com/ckan/ckan-solr). The index data is stored in a named volume and has a spatial schema. [^2] * Redis: standard Redis image * NGINX: latest stable nginx image that includes SSL and Non-SSL endpoints @@ -50,7 +50,7 @@ Information about extensions installed in the `main` image. More info described > Switch branches to see the `roadmap` for other projects: [ckan-docker/branches](https://github.com/mjanez/ckan-docker/branches) -| **Element** | **Description** | **version** | **Status** | **DEV**[^2] | **PRO**[^3] | **Remarks** | +| **Element** | **Description** | **version** | **Status** | **DEV**[^3] | **PRO**[^4] | **Remarks** | |-------------|-----------------------------------------------------------------------------------------|-------------|------------------------------|---------|---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Core | [CKAN](https://github.com/mjanez/ckan-docker) | 2.9.8 | Completed | ✔️ | ✔️ | Stable installation for version 2.9.8 (Production & Dev images) via Docker Compose based on [official images](https://github.com/ckan/ckan-docker-base)). Initial configuration, basic customisation and operation guide. | | Core + | [Datastore](https://github.com/mjanez/ckan-docker) | 2.9.8 | Completed | ✔️ | ✔️ | Stable installation (Production & Dev images) via Docker Compose. | @@ -60,7 +60,7 @@ Information about extensions installed in the `main` image. More info described | Extension | [ckanext-geoview](https://github.com/ckan/ckanext-geoview) | 0.0.20 | Completed | ✔️ | ✔️ | Stable installation. | | Extension | [ckanext-spatial](https://github.com/ckan/ckanext-spatial) | 2.0.0 | Completed | ✔️ | ✔️ | Stable installation, necessary for the implementation of the Collector ([ogc_ckan](#recollector-ckan)) | | Extension | [ckanext-dcat](https://github.com/mjanez/ckanext-dcat) | 1.2.0 | Completed | ✔️ | ✔️ | Stable installation, include DCAT-AP 2.1 profile compatible with GeoDCAT-AP. | -| Extension | [ckanext-scheming](https://github.com/mjanez/ckanext-scheming) | 3.0.0 | WIP | ✔️ | ✔️ | Stable installation. Customised ckanext schema[^4] based on the [Spanish Metadata Core](https://datos.gob.es/es/doc-tags/nti-risp) with the aim of completing the minimum metadata elements included in the current datasets in accordance with [GeoDCAT-AP](https://semiceu.github.io/GeoDCAT-AP/releases/) and [INSPIRE](https://inspire.ec.europa.eu/about-inspire). | +| Extension | [ckanext-scheming](https://github.com/mjanez/ckanext-scheming) | 3.0.0 | WIP | ✔️ | ✔️ | Stable installation. Customised ckanext schema[^5] based on the [Spanish Metadata Core](https://datos.gob.es/es/doc-tags/nti-risp) with the aim of completing the minimum metadata elements included in the current datasets in accordance with [GeoDCAT-AP](https://semiceu.github.io/GeoDCAT-AP/releases/) and [INSPIRE](https://inspire.ec.europa.eu/about-inspire). | | Extension | [ckanext-resourcedictionary](https://github.com/OpenDataGIS/ckanext-resourcedictionary) | main | Completed | ✔️ | ✔️ | Stable installation. This extension extends the default CKAN Data Dictionary functionality by adding possibility to create data dictionary before actual data is uploaded to datastore. | | Extension | [ckanext-pages](https://github.com/ckan/ckanext-pages) | 0.5.1 | Completed | ✔️ | ✔️ | Stable installation. This extension gives you an easy way to add simple pages to CKAN. | | Extension | [ckanext-pdfview](https://github.com/ckan/ckanext-pdfview) | 0.0.8 | Completed | ✔️ | ✔️ | Stable installation. This extension provides a view plugin for PDF files using an html object tag. | @@ -370,8 +370,9 @@ Available components: * **ckan2pycsw**: Software to achieve interoperability with the open data portals based on CKAN. To do this, ckan2pycsw reads data from an instance using the CKAN API, generates ISO-19115/ISO-19139 metadata using [pygeometa](https://geopython.github.io/pygeometa/), or a custom schema that is based on a customized CKAN schema, and populates a [pycsw](https://pycsw.org/) instance that exposes the metadata using CSW and OAI-PMH. -[^1]: Contains fields needed for the [ckanext-spatial geo search](https://docs.ckan.org/projects/ckanext-spatial/en/latest/spatial-search.html) -[^2]: Development environment. -[^3]: Production environment. -[^4]: [ckan_geodcatap](https://github.com/mjanez/ckanext-scheming/blob/036b8c6503059e0d42b0eba180d5bd39205c64a3/ckanext/scheming/ckan_geodcatap.yaml), more info: https://github.com/mjanez/ckanext-scheming/pull/1 -[^5]: A fork of [COATNor/coat2pycsw](https://github.com/COATnor/coat2pycsw) that has been extended to meet the needs of harvesting GeoDCAT-AP metadata according to INSPIRE ISO19139. +[^1]: Official CKAN repo: https://github.com/ckan/ckan-docker-base +[^2]: Contains fields needed for the [ckanext-spatial geo search](https://docs.ckan.org/projects/ckanext-spatial/en/latest/spatial-search.html) +[^3]: Development environment. +[^4]: Production environment. +[^5]: [ckan_geodcatap](https://github.com/mjanez/ckanext-scheming/blob/036b8c6503059e0d42b0eba180d5bd39205c64a3/ckanext/scheming/ckan_geodcatap.yaml), more info: https://github.com/mjanez/ckanext-scheming/pull/1 +[^6]: A fork of [COATNor/coat2pycsw](https://github.com/COATnor/coat2pycsw) that has been extended to meet the needs of harvesting GeoDCAT-AP metadata according to INSPIRE ISO19139. diff --git a/ckan/Dockerfile b/ckan/Dockerfile index 1dff8d6..1b3552e 100644 --- a/ckan/Dockerfile +++ b/ckan/Dockerfile @@ -1,117 +1,13 @@ -FROM alpine:3.13 +FROM ghcr.io/mjanez/ckan-base-spatial:master -# Internals, you probably don't need to change these +# Set up environment variables ENV APP_DIR=/srv/app -ENV SRC_DIR=/srv/app/src -ENV CKAN_INI=${APP_DIR}/ckan.ini -ENV PIP_SRC=${SRC_DIR} -ENV CKAN_STORAGE_PATH=/var/lib/ckan -ENV GIT_URL=https://github.com/ckan/ckan.git ENV TZ=UTC RUN echo ${TZ} > /etc/timezone -# CKAN version to build -ENV GIT_BRANCH=ckan-2.9.8 -# Customize these on the .env file if needed -ENV CKAN_SITE_URL=http://localhost:5000 -ENV CKAN__PLUGINS image_view text_view recline_view datastore datapusher envvars - -# UWSGI options -ENV UWSGI_HARAKIRI=50 - -WORKDIR ${APP_DIR} - -# Install necessary packages to run CKAN -RUN apk add --no-cache tzdata \ - git \ - gettext \ - postgresql-client \ - python3 \ - libxml2 \ - libxslt \ - musl-dev \ - uwsgi \ - uwsgi-http \ - uwsgi-corerouter \ - uwsgi-python \ - py3-gevent \ - uwsgi-gevent \ - libmagic \ - curl \ - patch \ - sudo && \ - # Packages to build CKAN requirements and plugins - apk add --no-cache --virtual .build-deps \ - postgresql-dev \ - gcc \ - make \ - g++ \ - autoconf \ - automake \ - libtool \ - python3-dev \ - libxml2-dev \ - libxslt-dev \ - linux-headers \ - openssl-dev \ - libffi-dev \ - cargo \ - # Set up PROJ - geos-dev \ - proj \ - proj-util \ - proj-dev && \ - # Create SRC_DIR - mkdir -p ${SRC_DIR} && \ - # Install pip, supervisord and uwsgi - curl -o ${SRC_DIR}/get-pip.py https://bootstrap.pypa.io/get-pip.py && \ - python3 ${SRC_DIR}/get-pip.py && \ - pip3 install supervisor && \ - mkdir /etc/supervisord.d && \ - #pip wheel --wheel-dir=/wheels uwsgi gevent && \ - rm -rf ${SRC_DIR}/get-pip.py - -COPY setup/supervisord.conf /etc - -# Install CKAN -RUN pip3 install -e git+${GIT_URL}@${GIT_BRANCH}#egg=ckan && \ - cd ${SRC_DIR}/ckan && \ - cp who.ini ${APP_DIR} && \ - pip3 install -r requirement-setuptools.txt && \ - pip3 install --no-binary markdown -r requirements.txt && \ - # Install CKAN envvars to support loading config from environment variables - pip3 install -e git+https://github.com/okfn/ckanext-envvars.git#egg=ckanext-envvars && \ - # Create and update CKAN config - ckan generate config ${CKAN_INI} && \ - ckan config-tool ${CKAN_INI} "beaker.session.secret = " && \ - ckan config-tool ${CKAN_INI} "ckan.plugins = ${CKAN__PLUGINS}" - -# Create a local user and group to run the app -RUN addgroup -g 92 -S ckan && \ - adduser -u 92 -h /srv/app -s /bin/bash -D -G ckan ckan - -# Create local storage folder -RUN mkdir -p ${CKAN_STORAGE_PATH} && \ - chown -R ckan:ckan ${CKAN_STORAGE_PATH} - -COPY setup/prerun.py ${APP_DIR} -COPY setup/start_ckan.sh ${APP_DIR} -ADD https://raw.githubusercontent.com/ckan/ckan/${GIT_BRANCH}/wsgi.py ${APP_DIR} - -# Create entrypoint directory for children image scripts -ONBUILD RUN mkdir /docker-entrypoint.d - -RUN chown ckan -R /srv/app - -RUN addgroup sudo && \ - adduser ckan sudo - -EXPOSE 5000 - -HEALTHCHECK --interval=60s --timeout=5s --retries=5 CMD curl --fail http://localhost:5000/api/3/action/status_show || exit 1 # Make sure both files are not exactly the same RUN if ! [ /usr/share/zoneinfo/${TZ} -ef /etc/localtime ]; then \ - cp /usr/share/zoneinfo/${TZ} /etc/localtime ;\ + cp /usr/share/zoneinfo/${TZ} /etc/localtime ;\ fi ; # Extensions diff --git a/ckan/Dockerfile.dev b/ckan/Dockerfile.dev index 0f72975..6f2d24b 100644 --- a/ckan/Dockerfile.dev +++ b/ckan/Dockerfile.dev @@ -1,18 +1,13 @@ -# syntax = edrevo/dockerfile-plus -INCLUDE+ Dockerfile.spatial-vanilla +FROM ghcr.io/mjanez/ckan-base-spatial:master # Set up environment variables ENV APP_DIR=/srv/app ENV TZ=UTC RUN echo ${TZ} > /etc/timezone -# Set up PROJ -# PROJ4 for ckanext-spatial: https://github.com/pyproj4/pyproj/issues/582 // https://github.com/nickumia-reisys/ckanext-geodatagov/commit/ed61ffdd118e21e7a1db482c75738c3f261aa180 -RUN apk add geos-dev proj proj-util proj-dev - # Make sure both files are not exactly the same RUN if ! [ /usr/share/zoneinfo/${TZ} -ef /etc/localtime ]; then \ - cp /usr/share/zoneinfo/${TZ} /etc/localtime ;\ + cp /usr/share/zoneinfo/${TZ} /etc/localtime ;\ fi ; # Install any extensions needed by your CKAN instance diff --git a/ckan/Dockerfile.spatial-vanilla b/ckan/Dockerfile.spatial-vanilla deleted file mode 100644 index 2e4acb5..0000000 --- a/ckan/Dockerfile.spatial-vanilla +++ /dev/null @@ -1,112 +0,0 @@ -FROM alpine:3.13 - -# Internals, you probably don't need to change these -ENV APP_DIR=/srv/app -ENV SRC_DIR=/srv/app/src -ENV CKAN_INI=${APP_DIR}/ckan.ini -ENV PIP_SRC=${SRC_DIR} -ENV CKAN_STORAGE_PATH=/var/lib/ckan -ENV GIT_URL=https://github.com/ckan/ckan.git -ENV TZ=UTC -RUN echo ${TZ} > /etc/timezone -# CKAN version to build -ENV GIT_BRANCH=ckan-2.9.8 -# Customize these on the .env file if needed -ENV CKAN_SITE_URL=http://localhost:5000 -ENV CKAN__PLUGINS image_view text_view recline_view datastore datapusher envvars - -# UWSGI options -ENV UWSGI_HARAKIRI=50 - -WORKDIR ${APP_DIR} - -# Install necessary packages to run CKAN -RUN apk add --no-cache tzdata \ - git \ - gettext \ - postgresql-client \ - python3 \ - libxml2 \ - libxslt \ - musl-dev \ - uwsgi \ - uwsgi-http \ - uwsgi-corerouter \ - uwsgi-python \ - py3-gevent \ - uwsgi-gevent \ - libmagic \ - curl \ - patch \ - sudo && \ - # Packages to build CKAN requirements and plugins - apk add --no-cache --virtual .build-deps \ - postgresql-dev \ - gcc \ - make \ - g++ \ - autoconf \ - automake \ - libtool \ - python3-dev \ - libxml2-dev \ - libxslt-dev \ - linux-headers \ - openssl-dev \ - libffi-dev \ - cargo \ - # Set up PROJ - geos-dev \ - proj \ - proj-util \ - proj-dev && \ - # Create SRC_DIR - mkdir -p ${SRC_DIR} && \ - # Install pip, supervisord and uwsgi - curl -o ${SRC_DIR}/get-pip.py https://bootstrap.pypa.io/get-pip.py && \ - python3 ${SRC_DIR}/get-pip.py && \ - pip3 install supervisor && \ - mkdir /etc/supervisord.d && \ - #pip wheel --wheel-dir=/wheels uwsgi gevent && \ - rm -rf ${SRC_DIR}/get-pip.py - -COPY setup/supervisord.conf /etc - -# Install CKAN -RUN pip3 install -e git+${GIT_URL}@${GIT_BRANCH}#egg=ckan && \ - cd ${SRC_DIR}/ckan && \ - cp who.ini ${APP_DIR} && \ - pip3 install -r requirement-setuptools.txt && \ - pip3 install --no-binary markdown -r requirements.txt && \ - # Install CKAN envvars to support loading config from environment variables - pip3 install -e git+https://github.com/okfn/ckanext-envvars.git#egg=ckanext-envvars && \ - # Create and update CKAN config - ckan generate config ${CKAN_INI} && \ - ckan config-tool ${CKAN_INI} "beaker.session.secret = " && \ - ckan config-tool ${CKAN_INI} "ckan.plugins = ${CKAN__PLUGINS}" - -# Create a local user and group to run the app -RUN addgroup -g 92 -S ckan && \ - adduser -u 92 -h /srv/app -s /bin/bash -D -G ckan ckan - -# Create local storage folder -RUN mkdir -p ${CKAN_STORAGE_PATH} && \ - chown -R ckan:ckan ${CKAN_STORAGE_PATH} - -COPY setup/prerun.py ${APP_DIR} -COPY setup/start_ckan.sh ${APP_DIR} -ADD https://raw.githubusercontent.com/ckan/ckan/${GIT_BRANCH}/wsgi.py ${APP_DIR} - -# Create entrypoint directory for children image scripts -ONBUILD RUN mkdir /docker-entrypoint.d - -RUN chown ckan -R /srv/app - -RUN addgroup sudo && \ - adduser ckan sudo - -EXPOSE 5000 - -HEALTHCHECK --interval=60s --timeout=5s --retries=5 CMD curl --fail http://localhost:5000/api/3/action/status_show || exit 1 - -CMD ["/srv/app/start_ckan.sh"] \ No newline at end of file