Update README

Simplify install instructions (no need to choose a branch on
ckanext-harvest, just use master). Updated examples and paths.

Added contributing and license info.
This commit is contained in:
amercader 2015-06-10 15:57:28 +01:00
parent 5b4bff329f
commit 84126bb821
1 changed files with 36 additions and 28 deletions

View File

@ -32,25 +32,9 @@ Installation
ckan.harvest.mq.type = rabbitmq ckan.harvest.mq.type = rabbitmq
2. Install the extension into your python environment. 2. Install the extension into your python environment::
*Note:* Depending on the CKAN core version you are targeting you will need to (pyenv) $ pip install -e git+https://github.com/ckan/ckanext-harvest.git#egg=ckanext-harvest
use a different branch from the extension.
For a production site, use the `stable` branch, unless there is a specific
branch that targets the CKAN core version that you are using.
To target the latest CKAN core release::
(pyenv) $ pip install -e git+https://github.com/okfn/ckanext-harvest.git@stable#egg=ckanext-harvest
To target an old release (if a release branch exists, otherwise use `stable`)::
(pyenv) $ pip install -e git+https://github.com/okfn/ckanext-harvest.git@release-v1.8#egg=ckanext-harvest
To target CKAN `master`, use the extension `master` branch (ie no branch defined)::
(pyenv) $ pip install -e git+https://github.com/okfn/ckanext-harvest.git#egg=ckanext-harvest
3. Install the rest of python modules required by the extension:: 3. Install the rest of python modules required by the extension::
@ -418,8 +402,8 @@ interface:
Here you can also find other examples of custom harvesters: Here you can also find other examples of custom harvesters:
* https://github.com/okfn/ckanext-pdeu/tree/master/ckanext/pdeu/harvesters * https://github.com/ckan/ckanext-dcat/tree/master/ckanext/dcat/harvesters
* https://github.com/okfn/ckanext-inspire/ckanext/inspire/harvesters.py * https://github.com/ckan/ckanext-spatial/tree/master/ckanext/spatial/harvesters
Running the harvest jobs Running the harvest jobs
@ -466,7 +450,7 @@ have already installed and configured the harvesting extension (See
`Installation` if not). `Installation` if not).
Note: It is recommended to run the harvest process from a non-root user Note: It is recommended to run the harvest process from a non-root user
(generally the one you are running CKAN with). Replace the user `okfn` in the (generally the one you are running CKAN with). Replace the user `ckan` in the
following steps with the one you are using. following steps with the one you are using.
1. Install Supervisor:: 1. Install Supervisor::
@ -494,10 +478,10 @@ following steps with the one you are using.
[program:ckan_gather_consumer] [program:ckan_gather_consumer]
command=/var/lib/ckan/std/pyenv/bin/paster --plugin=ckanext-harvest harvester gather_consumer --config=/etc/ckan/std/std.ini command=/usr/lib//ckan/default/bin/paster --plugin=ckanext-harvest harvester gather_consumer --config=/etc/ckan/std/std.ini
; user that owns virtual environment. ; user that owns virtual environment.
user=okfn user=ckan
numprocs=1 numprocs=1
stdout_logfile=/var/log/ckan/std/gather_consumer.log stdout_logfile=/var/log/ckan/std/gather_consumer.log
@ -508,10 +492,10 @@ following steps with the one you are using.
[program:ckan_fetch_consumer] [program:ckan_fetch_consumer]
command=/var/lib/ckan/std/pyenv/bin/paster --plugin=ckanext-harvest harvester fetch_consumer --config=/etc/ckan/std/std.ini command=/usr/lib//ckan/default/bin/paster --plugin=ckanext-harvest harvester fetch_consumer --config=/etc/ckan/std/std.ini
; user that owns virtual environment. ; user that owns virtual environment.
user=okfn user=ckan
numprocs=1 numprocs=1
stdout_logfile=/var/log/ckan/std/fetch_consumer.log stdout_logfile=/var/log/ckan/std/fetch_consumer.log
@ -574,20 +558,44 @@ following steps with the one you are using.
that will run the `run` harvester command periodically. To do so, edit the cron table with that will run the `run` harvester command periodically. To do so, edit the cron table with
the following command (it may ask you to choose an editor):: the following command (it may ask you to choose an editor)::
sudo crontab -e -u okfn sudo crontab -e -u ckan
Note that we are running this command as the same user we configured the processes to be run with Note that we are running this command as the same user we configured the processes to be run with
(`okfn` in our example). (`ckan` in our example).
Paste this line into your crontab, again replacing the paths to paster and the ini file with yours:: Paste this line into your crontab, again replacing the paths to paster and the ini file with yours::
# m h dom mon dow command # m h dom mon dow command
*/15 * * * * /var/lib/ckan/std/pyenv/bin/paster --plugin=ckanext-harvest harvester run --config=/etc/ckan/std/std.ini */15 * * * * /usr/lib/ckan/default/bin/paster --plugin=ckanext-harvest harvester run --config=/etc/ckan/std/std.ini
This particular example will check for pending jobs every fifteen minutes. This particular example will check for pending jobs every fifteen minutes.
You can of course modify this periodicity, this `Wikipedia page <http://en.wikipedia.org/wiki/Cron#CRON_expression>`_ You can of course modify this periodicity, this `Wikipedia page <http://en.wikipedia.org/wiki/Cron#CRON_expression>`_
has a good overview of the crontab syntax. has a good overview of the crontab syntax.
Community
=========
* Developer mailing list: `ckan-dev@lists.okfn.org <http://lists.okfn.org/mailman/listinfo/ckan-dev>`_
* Developer IRC channel: `#ckan on irc.freenode.net <http://webchat.freenode.net/?channels=ckan>`_
* `Issue tracker <https://github.com/ckan/ckanext-harvest/issues>`_
Contributing
============
For contributing to ckanext-spatial or its documentation, follow the same
guidelines that apply to CKAN core, described in
`CONTRIBUTING <https://github.com/ckan/ckan/blob/master/CONTRIBUTING.rst>`_.
License
=======
This extension is open and licensed under the GNU Affero General Public License (AGPL) v3.0.
Its full text may be found at:
http://www.fsf.org/licensing/licenses/agpl-3.0.html
.. _Supervisor: http://supervisord.org .. _Supervisor: http://supervisord.org