Link to latest Docker work on ckan/ckan
This commit is contained in:
Ian Ward
parent
dca4f4b800
commit
61c7865193
284
README.md
284
README.md
|
|
@ -1,283 +1 @@
|
|||
ckan-docker
|
||||
===========
|
||||
|
||||
Developing and deploying CKAN with Docker
|
||||
|
||||
|
||||
|
||||
# Intro
|
||||
|
||||
Dockerfiles, Docker-compose service definition & Vagrantfile to develop & deploy CKAN, Postgres, Solr & datapusher using Docker.
|
||||
|
||||
Docker containers included:
|
||||
|
||||
- CKAN (should work with any version 2.x)
|
||||
- Postgres (Postgres 9.3 and PostGIS 2.1, CKAN datastore & spatial extension supported)
|
||||
- Solr (4.10.1, custom schemas & spatial extension supported)
|
||||
- Docker-compose _[optional]_ (to manage the containers)
|
||||
- Data _[optional]_ (to store Postgres data & CKAN FileStore separately)
|
||||
|
||||
|
||||
Other contrib containers:
|
||||
|
||||
- Nginx (1.7.6 official) as a caching reverse proxy
|
||||
- Datapusher
|
||||
|
||||
|
||||
# Requirements
|
||||
|
||||
|Name |Version |Comment |
|
||||
|:--------------|:-------------:|:----------------------------------------------|
|
||||
|Docker |>= 1.3 |works with Boot2docker 1.3 |
|
||||
|Docker-compose |>= 1.1.0 |on the host or with Dockerfile provided |
|
||||
|Vagrant |>= 1.6 |if you intend to use Vagrant |
|
||||
|OS |any |as long as you can run Docker 1.3 |
|
||||
|
||||
|
||||
|
||||
# Reference
|
||||
|
||||
## Structure
|
||||
|
||||
├── Dockerfile (CKAN Dockerfile)
|
||||
├── README.md
|
||||
├── Vagrantfile (CKAN Vagrantfile)
|
||||
├── _etc (config copied to /etc)
|
||||
│ ├── apache2
|
||||
│ ├── ckan
|
||||
│ ├── cron.d
|
||||
│ ├── my_init.d
|
||||
│ ├── postfix
|
||||
│ └── supervisor
|
||||
├── _service-provider (any service provider such as datapusher)
|
||||
│ └── datapusher
|
||||
├── _solr
|
||||
│ └── schema.xml (version specific & custom schema)
|
||||
├── _src (CKAN source code & extensions)
|
||||
│ ├── ckan
|
||||
│ └── ckanext-...
|
||||
├── docker
|
||||
│ ├── ckan
|
||||
│ ├── data
|
||||
│ ├── compose
|
||||
│ ├── nginx
|
||||
│ ├── insecure_key (baseimage insecure SSH key)
|
||||
│ ├── postgres
|
||||
│ └── solr
|
||||
├── docker-compose.yml (CKAN services definition)
|
||||
└── vagrant
|
||||
└── docker-host (Linux Docker host if required)
|
||||
|
||||
### Directories
|
||||
|
||||
_the content from the directories prefixed with `_` need to be edited / configured as required before building the Dockerfiles._
|
||||
|
||||
#### _etc
|
||||
contains configuration files that are copied to /etc in the container. see _etc/README
|
||||
|
||||
#### _solr
|
||||
contains your custom Solr schema (for your version of CKAN, & extensions installed). see _solr/README
|
||||
|
||||
#### _src
|
||||
contains your packages source code (CKAN & extensions). see _src/README
|
||||
|
||||
#### _service-provider
|
||||
contains any service providers (e.g. datapusher) with their Dockerfiles. see _service-provider/README.
|
||||
|
||||
#### docker
|
||||
contains the Dockerfiles and any supporting files
|
||||
|
||||
#### vagrant
|
||||
contains the Docker host if the host cannot run Docker containers natively (OS X & Windows)
|
||||
|
||||
|
||||
### Files
|
||||
|
||||
#### Dockerfiles
|
||||
|
||||
The Dockerfiles are currently based on `phusion/baseimage:0.9.16`.
|
||||
|
||||
SSH is supported using an insecure key which is enabled by default for development purposes. You should disable it in production use for obvious reasons.
|
||||
|
||||
[Read this to find out more about phusion baseimage](https://phusion.github.io/baseimage-docker/)
|
||||
|
||||
##### CKAN Dockerfile
|
||||
The app container runs the following services
|
||||
|
||||
- Apache
|
||||
- Postfix
|
||||
- Supervisor
|
||||
- Cron
|
||||
|
||||
##### Postgres Dockerfile
|
||||
The database container runs Postgres 9.3 and PostGIS 2.1.
|
||||
It supports the [datastore](http://docs.ckan.org/en/latest/maintaining/datastore.html) & [ckanext-spatial](https://github.com/ckan/ckanext-spatial)
|
||||
|
||||
##### Data Dockerfile
|
||||
The data container is optional but recommended. It exposes two volumes to store the Postgres data `($PGDATA)` & CKAN FileStore. This means you can recreate / app containers without losing your data.
|
||||
|
||||
##### Solr Dockerfile
|
||||
The Solr container runs version 4.10.1. This can easily be changed by customising SOLR_VERSION in the Dockerfile.
|
||||
|
||||
By detault the `schema.xml` of the upstream version (2.3) is copied in the container. This can be overriden at runtime by mounting it as a volume.
|
||||
This default path of the volume is `<path to>/_src/ckan/ckan/config/solr/schema.xml` so it mounts the schema corresponding to your version of CKAN.
|
||||
|
||||
For example for Docker-compose:
|
||||
|
||||
solr:
|
||||
build: docker/solr
|
||||
hostname: solr
|
||||
domainname: localdomain
|
||||
ports:
|
||||
- "8983:8983"
|
||||
volumes:
|
||||
- <path to>/_src/ckan/ckan/config/solr/schema.xml:/opt/solr/example/solr/ckan/conf/schema.xml
|
||||
|
||||
If you need a custom schema, put it in `<full path to>/_solr` and change the path in the docker-compose or vagrant file.
|
||||
|
||||
volumes:
|
||||
- <path to>/_solr/schema.xml:/opt/solr/example/solr/ckan/conf/schema.xml
|
||||
|
||||
|
||||
The container is cross version compatible. You need mount the appropriate `schema.xml` as a volume, or build a child image, which will copy the `schema.xml` next to your Dockerfile.
|
||||
|
||||
Read the [ckanext-spatial documentation](http://docs.ckan.org/projects/ckanext-spatial/en) to add the required fields to your Solr schema if you use ckanext-spatial
|
||||
|
||||
|
||||
##### Docker-compose Dockerfile
|
||||
The Docker-compose container runs the latests Docker-compose & Docker within a container.
|
||||
|
||||
The Docker socket needs to be mounted as a volume to control Docker on the host. A source folder must be mounted to access the docker-compose definition
|
||||
|
||||
see docker/compose/Readme to find out how to use
|
||||
|
||||
#### Vagrantfile
|
||||
Defines VMs provided by Docker, a Virtual Box docker-host is used if the host can't run Docker containers natively. This is an alternative to Boot2Docker.
|
||||
|
||||
#### docker-compose.yml
|
||||
Defines the set of services required to run CKAN. Read the [docker-compose.yml reference](http://docs.docker.com/compose/yml/) to understand and edit.
|
||||
|
||||
|
||||
|
||||
---
|
||||
# Usage
|
||||
|
||||
1. Clone your code in the `_src` directory (see _src/README)
|
||||
2. Clone the datapusher in `_service-provider` (see _service-provider/README)
|
||||
3. Set the full path of the volumes in docker-compose.yml
|
||||
4. Run `up` with Docker-compose or Vagrant
|
||||
|
||||
|
||||
## Using Docker-compose (recommended)
|
||||
|
||||
#### Option 1: Docker-compose is installed on the Docker host
|
||||
_If you have it installed, just type_
|
||||
|
||||
docker-compose up
|
||||
|
||||
#### Option 2: Using the docker-compose container
|
||||
_Otherwise, you can use the container provided_
|
||||
|
||||
Build the Docker-compose container
|
||||
|
||||
docker build --tag="dockercompose_container" docker/compose
|
||||
|
||||
Run it
|
||||
|
||||
docker run -it -d --name="dockercompose-ckan" -p 2375 -v /var/run/docker.sock:/tmp/docker.sock -v $(pwd):/src dockercompose_container
|
||||
|
||||
_In the Docker-compose container docker-compose won't work with relative path, because the mount namespace is different, you need to change the relative path to absolute path_
|
||||
|
||||
for example, change the `./`:
|
||||
|
||||
volumes:
|
||||
- ./_src:/usr/lib/ckan/default/src
|
||||
|
||||
to an absolute path to you ckan-docker directory: `/Users/username/git/ckan/ckan-docker/`
|
||||
|
||||
volumes:
|
||||
- /Users/username/git/ckan/ckan-docker/_src:/usr/lib/ckan/default/src
|
||||
|
||||
Build & Run the services defined in `docker-compose.yml`
|
||||
|
||||
docker exec -it dockercompose-ckan docker-compose up
|
||||
|
||||
If you are using boot2docker, add entries in your hosts file e.g. `192.168.59.103 ckan.localdomain`
|
||||
|
||||
You can now access CKAN at http://ckan.localdomain:8080/ (Apache) & http://ckan.localdomain/ (Ngnix)
|
||||
|
||||
## Using Vagrant
|
||||
|
||||
Build & run
|
||||
|
||||
vagrant up --provider=docker --no-parallel
|
||||
|
||||
You can now access CKAN at http://localhost:8080/ (Apache)
|
||||
|
||||
You can also SSH inside the container if you have left the `--enable-insecure-key` option in the run command.
|
||||
|
||||
vagrant ssh ckan
|
||||
|
||||
SSH insecure key can be disabled by removing the `--enable-insecure-key` option from the run command.
|
||||
|
||||
---
|
||||
## Running commands inside the container
|
||||
|
||||
The simplest thing to do is to use the `docker exec` command, for example:
|
||||
|
||||
docker exec -it src_ckan_1 /bin/bash
|
||||
|
||||
You can also SSH inside the container if you have left the `--enable-insecure-key` option in the run command.
|
||||
|
||||
ssh -i docker/insecure_key -p 2222 root@ckan.localdomain
|
||||
|
||||
SSH insecure key can be disabled by removing the `--enable-insecure-key` option from the run command.
|
||||
|
||||
## Managing Docker images & containers
|
||||
|
||||
You should use docker-compose to manage your containers & images, this will ensure they are started/stopped in order
|
||||
|
||||
If you want to quickly remove all untagged images:
|
||||
|
||||
docker images -q --filter "dangling=true" | xargs docker rmi
|
||||
|
||||
If you want to quickly remove all stopped containers
|
||||
|
||||
docker rm $(docker ps -a -q)
|
||||
|
||||
---
|
||||
## Developing CKAN
|
||||
|
||||
### Using paster serve instead of apache for development
|
||||
CKAN container starts Apache2 by default and the `ckan.site_url` port is set to `8080` in `50_configure`.
|
||||
You can override that permanently in the `custom_options.ini`, or manually in the container, for instance if you want to use paster in a development context.
|
||||
|
||||
Example (`paster serve --reload` in debug mode):
|
||||
|
||||
docker exec -it src_ckan_1 /bin/bash
|
||||
supervisorctl stop apache2
|
||||
sed -i -r 's/debug = false/debug = true/' $CKAN_CONFIG/$CONFIG_FILE
|
||||
sed -i -r 's/ckan.localdomain:8080/ckan.localdomain:5000/' $CKAN_CONFIG/$CONFIG_FILE
|
||||
$CKAN_HOME/bin/paster serve --reload $CKAN_CONFIG/$CONFIG_FILE
|
||||
|
||||
### Frontend development
|
||||
Front end development is also possible (see [Frontend development guidelines](http://docs.ckan.org/en/latest/contributing/frontend/))
|
||||
|
||||
Install frontend dependencies:
|
||||
|
||||
docker exec -it src_ckan_1 /bin/bash
|
||||
apt-get update
|
||||
apt-get install -y nodejs npm
|
||||
ln -s /usr/bin/nodejs /usr/bin/node
|
||||
source $CKAN_HOME/bin/activate
|
||||
cd $CKAN_HOME/
|
||||
npm install nodewatch less@1.3.3
|
||||
|
||||
Both examples show that development dependencies should only be installed in the containers when required. Since they are not part of the `Dockerfile` they do not persist and only serve the purpose of development. When they are no longuer needed the container can be rebuilt allowing to test the application in a production-like state.
|
||||
|
||||
---
|
||||
|
||||
# Sources
|
||||
- [Docker](https://www.docker.com)
|
||||
- [Docker-compose](http://docs.docker.com/compose/)
|
||||
- [Vagrant Docker provider](https://docs.vagrantup.com/v2/docker/index.html)
|
||||
This repository is no longer maintained. Please see the Docker configuration in the main https://github.com/ckan/ckan repo and images on https://hub.docker.com/r/ckan/
|
||||
|
|
|
|||
Loading…
Reference in New Issue