Update README
Simplify install instructions (no need to choose a branch on ckanext-harvest, just use master). Updated examples and paths. Added contributing and license info.
This commit is contained in:
amercader
parent
5b4bff329f
commit
84126bb821
64
README.rst
64
README.rst
|
|
@ -32,25 +32,9 @@ Installation
|
|||
ckan.harvest.mq.type = rabbitmq
|
||||
|
||||
|
||||
2. Install the extension into your python environment.
|
||||
2. Install the extension into your python environment::
|
||||
|
||||
*Note:* Depending on the CKAN core version you are targeting you will need to
|
||||
use a different branch from the extension.
|
||||
|
||||
For a production site, use the `stable` branch, unless there is a specific
|
||||
branch that targets the CKAN core version that you are using.
|
||||
|
||||
To target the latest CKAN core release::
|
||||
|
||||
(pyenv) $ pip install -e git+https://github.com/okfn/ckanext-harvest.git@stable#egg=ckanext-harvest
|
||||
|
||||
To target an old release (if a release branch exists, otherwise use `stable`)::
|
||||
|
||||
(pyenv) $ pip install -e git+https://github.com/okfn/ckanext-harvest.git@release-v1.8#egg=ckanext-harvest
|
||||
|
||||
To target CKAN `master`, use the extension `master` branch (ie no branch defined)::
|
||||
|
||||
(pyenv) $ pip install -e git+https://github.com/okfn/ckanext-harvest.git#egg=ckanext-harvest
|
||||
(pyenv) $ pip install -e git+https://github.com/ckan/ckanext-harvest.git#egg=ckanext-harvest
|
||||
|
||||
3. Install the rest of python modules required by the extension::
|
||||
|
||||
|
|
@ -418,8 +402,8 @@ interface:
|
|||
|
||||
Here you can also find other examples of custom harvesters:
|
||||
|
||||
* https://github.com/okfn/ckanext-pdeu/tree/master/ckanext/pdeu/harvesters
|
||||
* https://github.com/okfn/ckanext-inspire/ckanext/inspire/harvesters.py
|
||||
* https://github.com/ckan/ckanext-dcat/tree/master/ckanext/dcat/harvesters
|
||||
* https://github.com/ckan/ckanext-spatial/tree/master/ckanext/spatial/harvesters
|
||||
|
||||
|
||||
Running the harvest jobs
|
||||
|
|
@ -466,7 +450,7 @@ have already installed and configured the harvesting extension (See
|
|||
`Installation` if not).
|
||||
|
||||
Note: It is recommended to run the harvest process from a non-root user
|
||||
(generally the one you are running CKAN with). Replace the user `okfn` in the
|
||||
(generally the one you are running CKAN with). Replace the user `ckan` in the
|
||||
following steps with the one you are using.
|
||||
|
||||
1. Install Supervisor::
|
||||
|
|
@ -494,10 +478,10 @@ following steps with the one you are using.
|
|||
|
||||
[program:ckan_gather_consumer]
|
||||
|
||||
command=/var/lib/ckan/std/pyenv/bin/paster --plugin=ckanext-harvest harvester gather_consumer --config=/etc/ckan/std/std.ini
|
||||
command=/usr/lib//ckan/default/bin/paster --plugin=ckanext-harvest harvester gather_consumer --config=/etc/ckan/std/std.ini
|
||||
|
||||
; user that owns virtual environment.
|
||||
user=okfn
|
||||
user=ckan
|
||||
|
||||
numprocs=1
|
||||
stdout_logfile=/var/log/ckan/std/gather_consumer.log
|
||||
|
|
@ -508,10 +492,10 @@ following steps with the one you are using.
|
|||
|
||||
[program:ckan_fetch_consumer]
|
||||
|
||||
command=/var/lib/ckan/std/pyenv/bin/paster --plugin=ckanext-harvest harvester fetch_consumer --config=/etc/ckan/std/std.ini
|
||||
command=/usr/lib//ckan/default/bin/paster --plugin=ckanext-harvest harvester fetch_consumer --config=/etc/ckan/std/std.ini
|
||||
|
||||
; user that owns virtual environment.
|
||||
user=okfn
|
||||
user=ckan
|
||||
|
||||
numprocs=1
|
||||
stdout_logfile=/var/log/ckan/std/fetch_consumer.log
|
||||
|
|
@ -574,20 +558,44 @@ following steps with the one you are using.
|
|||
that will run the `run` harvester command periodically. To do so, edit the cron table with
|
||||
the following command (it may ask you to choose an editor)::
|
||||
|
||||
sudo crontab -e -u okfn
|
||||
sudo crontab -e -u ckan
|
||||
|
||||
Note that we are running this command as the same user we configured the processes to be run with
|
||||
(`okfn` in our example).
|
||||
(`ckan` in our example).
|
||||
|
||||
Paste this line into your crontab, again replacing the paths to paster and the ini file with yours::
|
||||
|
||||
# m h dom mon dow command
|
||||
*/15 * * * * /var/lib/ckan/std/pyenv/bin/paster --plugin=ckanext-harvest harvester run --config=/etc/ckan/std/std.ini
|
||||
*/15 * * * * /usr/lib/ckan/default/bin/paster --plugin=ckanext-harvest harvester run --config=/etc/ckan/std/std.ini
|
||||
|
||||
This particular example will check for pending jobs every fifteen minutes.
|
||||
You can of course modify this periodicity, this `Wikipedia page <http://en.wikipedia.org/wiki/Cron#CRON_expression>`_
|
||||
has a good overview of the crontab syntax.
|
||||
|
||||
Community
|
||||
=========
|
||||
|
||||
* Developer mailing list: `ckan-dev@lists.okfn.org <http://lists.okfn.org/mailman/listinfo/ckan-dev>`_
|
||||
* Developer IRC channel: `#ckan on irc.freenode.net <http://webchat.freenode.net/?channels=ckan>`_
|
||||
* `Issue tracker <https://github.com/ckan/ckanext-harvest/issues>`_
|
||||
|
||||
|
||||
Contributing
|
||||
============
|
||||
|
||||
For contributing to ckanext-spatial or its documentation, follow the same
|
||||
guidelines that apply to CKAN core, described in
|
||||
`CONTRIBUTING <https://github.com/ckan/ckan/blob/master/CONTRIBUTING.rst>`_.
|
||||
|
||||
|
||||
License
|
||||
=======
|
||||
|
||||
This extension is open and licensed under the GNU Affero General Public License (AGPL) v3.0.
|
||||
Its full text may be found at:
|
||||
|
||||
http://www.fsf.org/licensing/licenses/agpl-3.0.html
|
||||
|
||||
|
||||
.. _Supervisor: http://supervisord.org
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue