102 lines
2.7 KiB
ReStructuredText
102 lines
2.7 KiB
ReStructuredText
================================================
|
|
ckanext-harvest - Remote harvesting extension
|
|
================================================
|
|
|
|
This extension will contain all harvesting related code, now present
|
|
in ckan core, ckanext-dgu and ckanext-csw.
|
|
|
|
Dependencies
|
|
============
|
|
|
|
You will need ckan installed, as well as the ckanext-dgu and ckanext-csw
|
|
plugins activated.
|
|
|
|
|
|
Configuration
|
|
=============
|
|
|
|
The extension needs a user with sysadmin privileges to perform the
|
|
harvesting jobs. You can create such a user running these two commands in
|
|
the ckan directory::
|
|
|
|
paster user add harvest
|
|
|
|
paster sysadmin add harvest
|
|
|
|
The user's API key must be defined in the CKAN
|
|
configuration file (.ini) in the [app:main] section::
|
|
|
|
ckan.harvesting.api_key = 4e1dac58-f642-4e54-bbc4-3ea262271fe2
|
|
|
|
|
|
The API URL used can be also defined in the ini file (it defaults to
|
|
http://localhost:5000/)::
|
|
|
|
ckan.api_url = <api_url>
|
|
|
|
If you are using the spatial search feature, you can define the projection
|
|
in which extents are stored in the database with the following option. Use
|
|
the EPSG code as an integer (e.g 4326, 4258, 27700, etc). It defaults to
|
|
4258::
|
|
|
|
ckan.harvesting.srid = 4258
|
|
|
|
|
|
|
|
Command line interface
|
|
======================
|
|
|
|
The following operations can be run from the command line using the
|
|
``paster harvester`` command::
|
|
|
|
harvester source {url} [{user-ref} [{publisher-ref}]]
|
|
- create new harvest source
|
|
|
|
harvester rmsource {url}
|
|
- remove a harvester source (and associated jobs)
|
|
|
|
harvester sources
|
|
- lists harvest sources
|
|
|
|
harvester job {source-id} [{user-ref}]
|
|
- create new harvesting job
|
|
|
|
harvester rmjob {job-id}
|
|
- remove a harvesting job
|
|
|
|
harvester jobs
|
|
- lists harvesting jobs
|
|
|
|
harvester run
|
|
- runs harvesting jobs
|
|
|
|
harvester extents
|
|
- creates or updates the extent geometry column for packages with
|
|
a bounding box defined in extras
|
|
|
|
The commands should be run from the ckanext-harvest directory and expect
|
|
a development.ini file to be present. Most of the time you will specify
|
|
the config explicitly though::
|
|
|
|
paster harvester sources --config=../ckan/development.ini
|
|
|
|
|
|
API
|
|
===
|
|
|
|
The extension adds the following call to the CKAN search API, which returns
|
|
packages with an extent that intersects with the bounding box provided::
|
|
|
|
/api/2/search/package/geo?bbox={minx,miny,maxx,maxy}[&crs={srid}]
|
|
|
|
If the bounding box coordinates are not in the same projection as the one
|
|
defined in the database, a CRS must be provided, in one of the following
|
|
forms:
|
|
|
|
- urn:ogc:def:crs:EPSG::4258
|
|
- EPSG:4258
|
|
- 4258
|
|
|
|
|
|
|