Link to latest Docker work on ckan/ckan
This commit is contained in:
Ian Ward
parent
dca4f4b800
commit
61c7865193
284
README.md
284
README.md
|
|
@ -1,283 +1 @@
|
||||||
ckan-docker
|
This repository is no longer maintained. Please see the Docker configuration in the main https://github.com/ckan/ckan repo and images on https://hub.docker.com/r/ckan/
|
||||||
===========
|
|
||||||
|
|
||||||
Developing and deploying CKAN with Docker
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
# Intro
|
|
||||||
|
|
||||||
Dockerfiles, Docker-compose service definition & Vagrantfile to develop & deploy CKAN, Postgres, Solr & datapusher using Docker.
|
|
||||||
|
|
||||||
Docker containers included:
|
|
||||||
|
|
||||||
- CKAN (should work with any version 2.x)
|
|
||||||
- Postgres (Postgres 9.3 and PostGIS 2.1, CKAN datastore & spatial extension supported)
|
|
||||||
- Solr (4.10.1, custom schemas & spatial extension supported)
|
|
||||||
- Docker-compose _[optional]_ (to manage the containers)
|
|
||||||
- Data _[optional]_ (to store Postgres data & CKAN FileStore separately)
|
|
||||||
|
|
||||||
|
|
||||||
Other contrib containers:
|
|
||||||
|
|
||||||
- Nginx (1.7.6 official) as a caching reverse proxy
|
|
||||||
- Datapusher
|
|
||||||
|
|
||||||
|
|
||||||
# Requirements
|
|
||||||
|
|
||||||
|Name |Version |Comment |
|
|
||||||
|:--------------|:-------------:|:----------------------------------------------|
|
|
||||||
|Docker |>= 1.3 |works with Boot2docker 1.3 |
|
|
||||||
|Docker-compose |>= 1.1.0 |on the host or with Dockerfile provided |
|
|
||||||
|Vagrant |>= 1.6 |if you intend to use Vagrant |
|
|
||||||
|OS |any |as long as you can run Docker 1.3 |
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
# Reference
|
|
||||||
|
|
||||||
## Structure
|
|
||||||
|
|
||||||
├── Dockerfile (CKAN Dockerfile)
|
|
||||||
├── README.md
|
|
||||||
├── Vagrantfile (CKAN Vagrantfile)
|
|
||||||
├── _etc (config copied to /etc)
|
|
||||||
│ ├── apache2
|
|
||||||
│ ├── ckan
|
|
||||||
│ ├── cron.d
|
|
||||||
│ ├── my_init.d
|
|
||||||
│ ├── postfix
|
|
||||||
│ └── supervisor
|
|
||||||
├── _service-provider (any service provider such as datapusher)
|
|
||||||
│ └── datapusher
|
|
||||||
├── _solr
|
|
||||||
│ └── schema.xml (version specific & custom schema)
|
|
||||||
├── _src (CKAN source code & extensions)
|
|
||||||
│ ├── ckan
|
|
||||||
│ └── ckanext-...
|
|
||||||
├── docker
|
|
||||||
│ ├── ckan
|
|
||||||
│ ├── data
|
|
||||||
│ ├── compose
|
|
||||||
│ ├── nginx
|
|
||||||
│ ├── insecure_key (baseimage insecure SSH key)
|
|
||||||
│ ├── postgres
|
|
||||||
│ └── solr
|
|
||||||
├── docker-compose.yml (CKAN services definition)
|
|
||||||
└── vagrant
|
|
||||||
└── docker-host (Linux Docker host if required)
|
|
||||||
|
|
||||||
### Directories
|
|
||||||
|
|
||||||
_the content from the directories prefixed with `_` need to be edited / configured as required before building the Dockerfiles._
|
|
||||||
|
|
||||||
#### _etc
|
|
||||||
contains configuration files that are copied to /etc in the container. see _etc/README
|
|
||||||
|
|
||||||
#### _solr
|
|
||||||
contains your custom Solr schema (for your version of CKAN, & extensions installed). see _solr/README
|
|
||||||
|
|
||||||
#### _src
|
|
||||||
contains your packages source code (CKAN & extensions). see _src/README
|
|
||||||
|
|
||||||
#### _service-provider
|
|
||||||
contains any service providers (e.g. datapusher) with their Dockerfiles. see _service-provider/README.
|
|
||||||
|
|
||||||
#### docker
|
|
||||||
contains the Dockerfiles and any supporting files
|
|
||||||
|
|
||||||
#### vagrant
|
|
||||||
contains the Docker host if the host cannot run Docker containers natively (OS X & Windows)
|
|
||||||
|
|
||||||
|
|
||||||
### Files
|
|
||||||
|
|
||||||
#### Dockerfiles
|
|
||||||
|
|
||||||
The Dockerfiles are currently based on `phusion/baseimage:0.9.16`.
|
|
||||||
|
|
||||||
SSH is supported using an insecure key which is enabled by default for development purposes. You should disable it in production use for obvious reasons.
|
|
||||||
|
|
||||||
[Read this to find out more about phusion baseimage](https://phusion.github.io/baseimage-docker/)
|
|
||||||
|
|
||||||
##### CKAN Dockerfile
|
|
||||||
The app container runs the following services
|
|
||||||
|
|
||||||
- Apache
|
|
||||||
- Postfix
|
|
||||||
- Supervisor
|
|
||||||
- Cron
|
|
||||||
|
|
||||||
##### Postgres Dockerfile
|
|
||||||
The database container runs Postgres 9.3 and PostGIS 2.1.
|
|
||||||
It supports the [datastore](http://docs.ckan.org/en/latest/maintaining/datastore.html) & [ckanext-spatial](https://github.com/ckan/ckanext-spatial)
|
|
||||||
|
|
||||||
##### Data Dockerfile
|
|
||||||
The data container is optional but recommended. It exposes two volumes to store the Postgres data `($PGDATA)` & CKAN FileStore. This means you can recreate / app containers without losing your data.
|
|
||||||
|
|
||||||
##### Solr Dockerfile
|
|
||||||
The Solr container runs version 4.10.1. This can easily be changed by customising SOLR_VERSION in the Dockerfile.
|
|
||||||
|
|
||||||
By detault the `schema.xml` of the upstream version (2.3) is copied in the container. This can be overriden at runtime by mounting it as a volume.
|
|
||||||
This default path of the volume is `<path to>/_src/ckan/ckan/config/solr/schema.xml` so it mounts the schema corresponding to your version of CKAN.
|
|
||||||
|
|
||||||
For example for Docker-compose:
|
|
||||||
|
|
||||||
solr:
|
|
||||||
build: docker/solr
|
|
||||||
hostname: solr
|
|
||||||
domainname: localdomain
|
|
||||||
ports:
|
|
||||||
- "8983:8983"
|
|
||||||
volumes:
|
|
||||||
- <path to>/_src/ckan/ckan/config/solr/schema.xml:/opt/solr/example/solr/ckan/conf/schema.xml
|
|
||||||
|
|
||||||
If you need a custom schema, put it in `<full path to>/_solr` and change the path in the docker-compose or vagrant file.
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
- <path to>/_solr/schema.xml:/opt/solr/example/solr/ckan/conf/schema.xml
|
|
||||||
|
|
||||||
|
|
||||||
The container is cross version compatible. You need mount the appropriate `schema.xml` as a volume, or build a child image, which will copy the `schema.xml` next to your Dockerfile.
|
|
||||||
|
|
||||||
Read the [ckanext-spatial documentation](http://docs.ckan.org/projects/ckanext-spatial/en) to add the required fields to your Solr schema if you use ckanext-spatial
|
|
||||||
|
|
||||||
|
|
||||||
##### Docker-compose Dockerfile
|
|
||||||
The Docker-compose container runs the latests Docker-compose & Docker within a container.
|
|
||||||
|
|
||||||
The Docker socket needs to be mounted as a volume to control Docker on the host. A source folder must be mounted to access the docker-compose definition
|
|
||||||
|
|
||||||
see docker/compose/Readme to find out how to use
|
|
||||||
|
|
||||||
#### Vagrantfile
|
|
||||||
Defines VMs provided by Docker, a Virtual Box docker-host is used if the host can't run Docker containers natively. This is an alternative to Boot2Docker.
|
|
||||||
|
|
||||||
#### docker-compose.yml
|
|
||||||
Defines the set of services required to run CKAN. Read the [docker-compose.yml reference](http://docs.docker.com/compose/yml/) to understand and edit.
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
---
|
|
||||||
# Usage
|
|
||||||
|
|
||||||
1. Clone your code in the `_src` directory (see _src/README)
|
|
||||||
2. Clone the datapusher in `_service-provider` (see _service-provider/README)
|
|
||||||
3. Set the full path of the volumes in docker-compose.yml
|
|
||||||
4. Run `up` with Docker-compose or Vagrant
|
|
||||||
|
|
||||||
|
|
||||||
## Using Docker-compose (recommended)
|
|
||||||
|
|
||||||
#### Option 1: Docker-compose is installed on the Docker host
|
|
||||||
_If you have it installed, just type_
|
|
||||||
|
|
||||||
docker-compose up
|
|
||||||
|
|
||||||
#### Option 2: Using the docker-compose container
|
|
||||||
_Otherwise, you can use the container provided_
|
|
||||||
|
|
||||||
Build the Docker-compose container
|
|
||||||
|
|
||||||
docker build --tag="dockercompose_container" docker/compose
|
|
||||||
|
|
||||||
Run it
|
|
||||||
|
|
||||||
docker run -it -d --name="dockercompose-ckan" -p 2375 -v /var/run/docker.sock:/tmp/docker.sock -v $(pwd):/src dockercompose_container
|
|
||||||
|
|
||||||
_In the Docker-compose container docker-compose won't work with relative path, because the mount namespace is different, you need to change the relative path to absolute path_
|
|
||||||
|
|
||||||
for example, change the `./`:
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
- ./_src:/usr/lib/ckan/default/src
|
|
||||||
|
|
||||||
to an absolute path to you ckan-docker directory: `/Users/username/git/ckan/ckan-docker/`
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
- /Users/username/git/ckan/ckan-docker/_src:/usr/lib/ckan/default/src
|
|
||||||
|
|
||||||
Build & Run the services defined in `docker-compose.yml`
|
|
||||||
|
|
||||||
docker exec -it dockercompose-ckan docker-compose up
|
|
||||||
|
|
||||||
If you are using boot2docker, add entries in your hosts file e.g. `192.168.59.103 ckan.localdomain`
|
|
||||||
|
|
||||||
You can now access CKAN at http://ckan.localdomain:8080/ (Apache) & http://ckan.localdomain/ (Ngnix)
|
|
||||||
|
|
||||||
## Using Vagrant
|
|
||||||
|
|
||||||
Build & run
|
|
||||||
|
|
||||||
vagrant up --provider=docker --no-parallel
|
|
||||||
|
|
||||||
You can now access CKAN at http://localhost:8080/ (Apache)
|
|
||||||
|
|
||||||
You can also SSH inside the container if you have left the `--enable-insecure-key` option in the run command.
|
|
||||||
|
|
||||||
vagrant ssh ckan
|
|
||||||
|
|
||||||
SSH insecure key can be disabled by removing the `--enable-insecure-key` option from the run command.
|
|
||||||
|
|
||||||
---
|
|
||||||
## Running commands inside the container
|
|
||||||
|
|
||||||
The simplest thing to do is to use the `docker exec` command, for example:
|
|
||||||
|
|
||||||
docker exec -it src_ckan_1 /bin/bash
|
|
||||||
|
|
||||||
You can also SSH inside the container if you have left the `--enable-insecure-key` option in the run command.
|
|
||||||
|
|
||||||
ssh -i docker/insecure_key -p 2222 root@ckan.localdomain
|
|
||||||
|
|
||||||
SSH insecure key can be disabled by removing the `--enable-insecure-key` option from the run command.
|
|
||||||
|
|
||||||
## Managing Docker images & containers
|
|
||||||
|
|
||||||
You should use docker-compose to manage your containers & images, this will ensure they are started/stopped in order
|
|
||||||
|
|
||||||
If you want to quickly remove all untagged images:
|
|
||||||
|
|
||||||
docker images -q --filter "dangling=true" | xargs docker rmi
|
|
||||||
|
|
||||||
If you want to quickly remove all stopped containers
|
|
||||||
|
|
||||||
docker rm $(docker ps -a -q)
|
|
||||||
|
|
||||||
---
|
|
||||||
## Developing CKAN
|
|
||||||
|
|
||||||
### Using paster serve instead of apache for development
|
|
||||||
CKAN container starts Apache2 by default and the `ckan.site_url` port is set to `8080` in `50_configure`.
|
|
||||||
You can override that permanently in the `custom_options.ini`, or manually in the container, for instance if you want to use paster in a development context.
|
|
||||||
|
|
||||||
Example (`paster serve --reload` in debug mode):
|
|
||||||
|
|
||||||
docker exec -it src_ckan_1 /bin/bash
|
|
||||||
supervisorctl stop apache2
|
|
||||||
sed -i -r 's/debug = false/debug = true/' $CKAN_CONFIG/$CONFIG_FILE
|
|
||||||
sed -i -r 's/ckan.localdomain:8080/ckan.localdomain:5000/' $CKAN_CONFIG/$CONFIG_FILE
|
|
||||||
$CKAN_HOME/bin/paster serve --reload $CKAN_CONFIG/$CONFIG_FILE
|
|
||||||
|
|
||||||
### Frontend development
|
|
||||||
Front end development is also possible (see [Frontend development guidelines](http://docs.ckan.org/en/latest/contributing/frontend/))
|
|
||||||
|
|
||||||
Install frontend dependencies:
|
|
||||||
|
|
||||||
docker exec -it src_ckan_1 /bin/bash
|
|
||||||
apt-get update
|
|
||||||
apt-get install -y nodejs npm
|
|
||||||
ln -s /usr/bin/nodejs /usr/bin/node
|
|
||||||
source $CKAN_HOME/bin/activate
|
|
||||||
cd $CKAN_HOME/
|
|
||||||
npm install nodewatch less@1.3.3
|
|
||||||
|
|
||||||
Both examples show that development dependencies should only be installed in the containers when required. Since they are not part of the `Dockerfile` they do not persist and only serve the purpose of development. When they are no longuer needed the container can be rebuilt allowing to test the application in a production-like state.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
# Sources
|
|
||||||
- [Docker](https://www.docker.com)
|
|
||||||
- [Docker-compose](http://docs.docker.com/compose/)
|
|
||||||
- [Vagrant Docker provider](https://docs.vagrantup.com/v2/docker/index.html)
|
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue