UPDATED:
- CKAN Dockerfile - improved caching of common requirements with the Datapusher - Added $CKAN_INI env variable - downgraded to pip 1.4.1 because 1.5 has a bug with HTTPS proxies https://github.com/pypa/pip/issues/1805 - fixed incorrect default port exposed - Fig Dockerfile to 1.0.1 - fig.yml - mount the custom_options.ini as a volume - use relative path by default - Documentation ADDED: - Datapusher Dockerfile based on fusion baseimage.
This commit is contained in:
Clément MOUCHET
parent
7d2783cbbe
commit
9a20dea6a4
29
Dockerfile
29
Dockerfile
|
|
@ -7,13 +7,6 @@ RUN locale-gen en_US.UTF-8 && \
|
|||
|
||||
RUN apt-get -qq update
|
||||
|
||||
ENV HOME /root
|
||||
ENV CKAN_HOME /usr/lib/ckan/default
|
||||
ENV CKAN_CONFIG /etc/ckan/default
|
||||
ENV CONFIG_FILE ckan.ini
|
||||
ENV CONFIG_OPTIONS custom_options.ini
|
||||
ENV CKAN_DATA /var/lib/ckan
|
||||
|
||||
# Install required packages
|
||||
RUN DEBIAN_FRONTEND=noninteractive apt-get -qq -y install \
|
||||
python-minimal \
|
||||
|
|
@ -21,16 +14,27 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get -qq -y install \
|
|||
python-virtualenv \
|
||||
libevent-dev \
|
||||
libpq-dev \
|
||||
libxml2-dev \
|
||||
libxslt1-dev \
|
||||
build-essential
|
||||
|
||||
# Install required packages
|
||||
RUN DEBIAN_FRONTEND=noninteractive apt-get -qq -y install \
|
||||
apache2 \
|
||||
libapache2-mod-wsgi \
|
||||
postfix \
|
||||
build-essential \
|
||||
git \
|
||||
libxml2-dev \
|
||||
libxslt1-dev \
|
||||
libgeos-c1 \
|
||||
supervisor
|
||||
|
||||
ENV HOME /root
|
||||
ENV CKAN_HOME /usr/lib/ckan/default
|
||||
ENV CKAN_CONFIG /etc/ckan/default
|
||||
ENV CONFIG_FILE ckan.ini
|
||||
ENV CONFIG_OPTIONS custom_options.ini
|
||||
ENV CKAN_DATA /var/lib/ckan
|
||||
ENV CKAN_INI $CKAN_CONFIG/$CONFIG_FILE
|
||||
|
||||
# Create directories & virtual env for CKAN
|
||||
RUN virtualenv $CKAN_HOME
|
||||
RUN mkdir -p $CKAN_CONFIG $CKAN_DATA /var/log/ckan
|
||||
|
|
@ -42,6 +46,7 @@ ADD docker/ckan/pip_install_req.sh /usr/local/sbin/pip_install_req
|
|||
# copy CKAN and any extenstions in the source directory
|
||||
ADD _src/ $CKAN_HOME/src/
|
||||
ONBUILD ADD _src/ $CKAN_HOME/src/
|
||||
RUN $CKAN_HOME/bin/pip install pip==1.4.1
|
||||
# install what we've just copied
|
||||
RUN pip_install_req
|
||||
ONBUILD RUN pip_install_req
|
||||
|
|
@ -74,7 +79,9 @@ ONBUILD COPY _etc/supervisor/conf.d/ /etc/supervisor/conf.d/
|
|||
|
||||
# Configure cron
|
||||
COPY _etc/cron.d/ /etc/cron.d/
|
||||
RUN chmod 600 -R /etc/cron.d/
|
||||
ONBUILD COPY _etc/cron.d/ /etc/cron.d/
|
||||
ONBUILD RUN chmod 600 -R /etc/cron.d/
|
||||
|
||||
# Configure runit
|
||||
ADD docker/ckan/my_init.d/ /etc/my_init.d/
|
||||
|
|
@ -84,7 +91,7 @@ ADD docker/ckan/svc/ /etc/service/
|
|||
CMD ["/sbin/my_init"]
|
||||
|
||||
VOLUME ["/usr/lib/ckan", "/etc/ckan"]
|
||||
EXPOSE 80 8800
|
||||
EXPOSE 8080
|
||||
|
||||
RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
|
||||
|
||||
|
|
|
|||
56
README.md
56
README.md
|
|
@ -81,7 +81,6 @@ contains your packages source code (CKAN & extensions). see _src/README
|
|||
#### _service-provider
|
||||
contains any service providers (e.g. datapusher) with their Dockerfiles. see _service-provider/README.
|
||||
|
||||
|
||||
#### docker
|
||||
contains the Dockerfiles and any supporting files
|
||||
|
||||
|
|
@ -93,9 +92,9 @@ contains the Docker host if the host cannot run Docker containers natively (OS X
|
|||
|
||||
#### Dockerfiles
|
||||
|
||||
The Dockerfiles are currently based on `phusion/baseimage:0.9.15`.
|
||||
The Dockerfiles are currently based on `phusion/baseimage:0.9.15`.
|
||||
|
||||
SSH is supported using an insecure key which is enabled by default for development purposes. You should disable it in production use for obvious reasons.
|
||||
SSH is supported using an insecure key which is enabled by default for development purposes. You should disable it in production use for obvious reasons.
|
||||
|
||||
[Read this to find out more about phusion baseimage](https://phusion.github.io/baseimage-docker/)
|
||||
|
||||
|
|
@ -140,7 +139,7 @@ Read the [ckanext-spatial documentation](http://docs.ckan.org/projects/ckanext-s
|
|||
|
||||
|
||||
##### Fig Dockerfile
|
||||
The Fig container runs Fig version 1.0 & the latest Docker within a container.
|
||||
The Fig container runs Fig version `1.0.1` & the latest Docker within a container.
|
||||
|
||||
The Docker socket needs to be mounted as a volume to control Docker on the host. A source folder must be mounted to access the fig definition
|
||||
|
||||
|
|
@ -165,11 +164,13 @@ Defines the set of services required to run CKAN. Read the [fig.yml reference](h
|
|||
|
||||
## Using Fig (recommended)
|
||||
|
||||
#### Option 1: Fig is installed on the Docker host
|
||||
_If you have if >= 1.0 installed, just type_
|
||||
|
||||
fig up
|
||||
|
||||
_Otherwise, install Fig on your host or use the container provided_
|
||||
#### Option 2: Using the fig container
|
||||
_Otherwise, you can use the container provided_
|
||||
|
||||
Build fig the fig container
|
||||
|
||||
|
|
@ -177,11 +178,23 @@ Build fig the fig container
|
|||
|
||||
Run it
|
||||
|
||||
docker run -it -d --name="fig-cli" -p 2375 -v /var/run/docker.sock:/tmp/docker.sock -v $(pwd):/src fig_container
|
||||
docker run -it -d --name="fig-ckan" -p 2375 -v /var/run/docker.sock:/tmp/docker.sock -v $(pwd):/src fig_container
|
||||
|
||||
_In the fig container fig won't work with relative path, because the mount namespace is different, you need to change the relative path to absolute path_
|
||||
|
||||
for example, change the `./`:
|
||||
|
||||
volumes:
|
||||
- ./_src:/usr/lib/ckan/default/src
|
||||
|
||||
to an absolute path to you ckan-docker directory: `/Users/username/git/ckan/ckan-docker/`
|
||||
|
||||
volumes:
|
||||
- /Users/username/git/ckan/ckan-docker/_src:/usr/lib/ckan/default/src
|
||||
|
||||
Build & Run the services defined in `fig.yml`
|
||||
|
||||
docker exec -it fig-cli fig up
|
||||
docker exec -it fig-ckan fig up
|
||||
|
||||
If you are using boot2docker, add entries in your hosts file e.g. `192.168.59.103 ckan.localdomain`
|
||||
|
||||
|
|
@ -226,6 +239,35 @@ If you want to quickly remove all stopped containers
|
|||
|
||||
docker rm $(docker ps -a -q)
|
||||
|
||||
---
|
||||
## Developping CKAN
|
||||
|
||||
### Using paster serve instead of apache for development
|
||||
CKAN container starts Apache2 by default and the `ckan.site_url` port is set to `8080` in `50_configure`.
|
||||
You can override that permanently in the `custom_options.ini`, or manually in the container, for instance if you want to use paster in a development context.
|
||||
|
||||
Example (`paster serve --reload` in debug mode):
|
||||
|
||||
docker exec -it src_ckan_1 /bin/bash
|
||||
supervisorctl stop apache2
|
||||
sed -i -r 's/debug = false/debug = true/' $CKAN_CONFIG/$CONFIG_FILE
|
||||
sed -i -r 's/ckan.localdomain:8080/ckan.localdomain:5000/' $CKAN_CONFIG/$CONFIG_FILE
|
||||
$CKAN_HOME/bin/paster serve --reload $CKAN_CONFIG/$CONFIG_FILE
|
||||
|
||||
### Frontend development
|
||||
Front end development is also possible (see [Frontend development guidelines](http://docs.ckan.org/en/latest/contributing/frontend/))
|
||||
|
||||
Install frontend dependencies:
|
||||
|
||||
docker exec -it src_ckan_1 /bin/bash
|
||||
apt-get update
|
||||
apt-get install -y nodejs npm
|
||||
ln -s /usr/bin/nodejs /usr/bin/node
|
||||
source $CKAN_HOME/bin/activate
|
||||
cd $CKAN_HOME/
|
||||
npm install nodewatch less@1.3.3
|
||||
|
||||
Both examples show that development dependencies should only be installed in the containers when required. Since they are not part of the `Dockerfile` they do not persist and only serve the purpose of development. When they are no longuer needed the container can be rebuilt allowing to test the application in a production-like state.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -6,7 +6,11 @@
|
|||
nodaemon=true
|
||||
|
||||
[program:apache2]
|
||||
command=/bin/bash -c "source /etc/apache2/envvars && exec /usr/sbin/apache2 -DFOREGROUND"
|
||||
command = /bin/bash -c "source /etc/apache2/envvars && exec /usr/sbin/apache2 -DFOREGROUND"
|
||||
|
||||
[program:postfix]
|
||||
command=/bin/bash -c "service postfix start && tail -f /var/log/mail.log"
|
||||
process_name = master
|
||||
directory = /etc/postfix
|
||||
command = /usr/sbin/postfix -c /etc/postfix start
|
||||
startsecs = 0
|
||||
autorestart = false
|
||||
|
|
|
|||
|
|
@ -3,34 +3,21 @@ _service-provider
|
|||
|
||||
_Clone any service providers with their Docker files here if you want to build their container._
|
||||
|
||||
<br>
|
||||
|
||||
## datapusher
|
||||
|
||||
For example, if you want to build the datapusher service yourself:
|
||||
|
||||
Clone datapusher:
|
||||
Clone datapusher in `_datapusher`:
|
||||
|
||||
git clone https://github.com/clementmouchet/datapusher.git
|
||||
git clone https://github.com/ckan/datapusher.git
|
||||
|
||||
|
||||
Tell the fig file to build it:
|
||||
|
||||
datapusher:
|
||||
build: _service-provider/datapusher
|
||||
hostname: datapusher
|
||||
domainname: localdomain
|
||||
ports:
|
||||
- "8800:8800"
|
||||
build: _service-provider/_datapusher
|
||||
hostname: datapusher
|
||||
domainname: localdomain
|
||||
ports:
|
||||
- "8800:8800"
|
||||
|
||||
<br>
|
||||
|
||||
If you do not want to build it, you can pull the image from the Docker registry. In this case, simply edit the fig file and specify the image instead of the build directory.
|
||||
|
||||
|
||||
datapusher:
|
||||
image: clementmouchet/datapusher
|
||||
hostname: datapusher
|
||||
domainname: localdomain
|
||||
ports:
|
||||
- "8800:8800"
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ RUN apt-get -qq update
|
|||
RUN DEBIAN_FRONTEND=noninteractive apt-get -qq -y install curl
|
||||
|
||||
RUN curl -sSL https://get.docker.com/ | sh
|
||||
RUN curl -L https://github.com/docker/fig/releases/download/1.0.0/fig-`uname -s`-`uname -m` > /usr/local/bin/fig; chmod +x /usr/local/bin/fig
|
||||
RUN curl -L https://github.com/docker/fig/releases/download/1.0.1/fig-`uname -s`-`uname -m` > /usr/local/bin/fig; chmod +x /usr/local/bin/fig
|
||||
ENV DOCKER_HOST unix:///tmp/docker.sock
|
||||
|
||||
WORKDIR /src
|
||||
|
|
|
|||
|
|
@ -10,32 +10,65 @@ The Docker socket needs to be mounted as a volume to control Docker on the host.
|
|||
|
||||
## Build & Run
|
||||
|
||||
_from your root directory of the project_
|
||||
_Move to the `ckan-docker` directory_
|
||||
|
||||
Build it:
|
||||
- Build it:
|
||||
|
||||
_Build the Docker image from `docker/fig/Dockerfile`, and call it "fig_container"_
|
||||
|
||||
docker build --tag="fig_container" docker/fig
|
||||
|
||||
Run it:
|
||||
- Run it:
|
||||
|
||||
docker run -it -d --name="fig-cli" -p 2375 -v /var/run/docker.sock:/tmp/docker.sock -v $(pwd):/src fig_container
|
||||
_Start the a container based on the `fig_container` Docker image_
|
||||
|
||||
docker run -it -d --name="fig-ckan" -p 2375 -v /var/run/docker.sock:/tmp/docker.sock -v $(pwd):/src fig_container
|
||||
|
||||
- Set the source volume path to yours.
|
||||
|
||||
_In the fig container fig won't work with relative path, because the mount namespace is different, you need to change the relative path to absolute path_
|
||||
|
||||
for example, change the `./`:
|
||||
|
||||
volumes:
|
||||
- ./_src:/usr/lib/ckan/default/src
|
||||
|
||||
to an absolute path to you ckan-docker directory: `/Users/username/git/ckan/ckan-docker/`
|
||||
|
||||
volumes:
|
||||
- /Users/username/git/ckan/ckan-docker/_src:/usr/lib/ckan/default/src
|
||||
|
||||
- Build the Docker images & run the containers:
|
||||
|
||||
Send the `fig up` command to the fig container
|
||||
|
||||
docker exec -it fig-ckan fig up
|
||||
|
||||
set the source volume path to yours
|
||||
|
||||
## Using the Fig container
|
||||
|
||||
### bring it back up
|
||||
You can use amy fig command by pre-pending the `docker exec -it fig-ckan` command to any fig command
|
||||
|
||||
docker exec -it fig-cli fig up
|
||||
### build & start / recreate
|
||||
|
||||
docker exec -it fig-ckan fig up
|
||||
|
||||
### stop it
|
||||
|
||||
docker exec -it fig-cli fig stop
|
||||
docker exec -it fig-ckan fig stop
|
||||
|
||||
### start it
|
||||
|
||||
docker exec -it fig-ckan fig start
|
||||
|
||||
### delete the containers
|
||||
|
||||
docker exec -it fig-cli fig rm
|
||||
docker exec -it fig-ckan fig rm
|
||||
|
||||
### build new images
|
||||
|
||||
docker exec -it fig-cli fig build
|
||||
docker exec -it fig-ckan fig build
|
||||
|
||||
### logs
|
||||
|
||||
docker exec -it fig-ckan fig logs
|
||||
|
|
|
|||
8
fig.yml
8
fig.yml
|
|
@ -15,7 +15,7 @@ solr:
|
|||
ports:
|
||||
- "8983:8983"
|
||||
volumes:
|
||||
- <path to>/ckan-docker/_src/ckan/ckan/config/solr/schema.xml:/opt/solr/example/solr/ckan/conf/schema.xml
|
||||
- ./_src/ckan/ckan/config/solr/schema.xml:/opt/solr/example/solr/ckan/conf/schema.xml
|
||||
|
||||
datapusher:
|
||||
build: _service-provider/datapusher
|
||||
|
|
@ -38,7 +38,9 @@ ckan:
|
|||
- datapusher:datapusher
|
||||
command: /sbin/my_init --enable-insecure-key
|
||||
volumes:
|
||||
- <path to>/ckan-docker/_src:/usr/lib/ckan/default/src
|
||||
- ./_src:/usr/lib/ckan/default/src
|
||||
- ./_etc/ckan/custom_options.ini:/etc/ckan/default/custom_options.ini
|
||||
- ./_etc/supervisor/conf.d:/etc/supervisor/conf.d
|
||||
|
||||
nginx:
|
||||
image: nginx:1.7.6
|
||||
|
|
@ -49,7 +51,7 @@ nginx:
|
|||
ports:
|
||||
- "80:80"
|
||||
volumes:
|
||||
- <path to>/ckan-docker/_etc/nginx/nginx.conf:/etc/nginx/nginx.conf
|
||||
- ./_etc/nginx/nginx.conf:/etc/nginx/nginx.conf
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue