Commit Graph

12 Commits

Author SHA1 Message Date
amercader 4964ce9ec3 [#8] Provide a config option to continue after validation errors
This can be set instance wide on the ini file with

ckanext.spatial.harvest.continue_on_validation_errors

or per source, adding continue_on_validation_errors=true to the source
config.
2013-02-15 14:47:25 +00:00
amercader 596bdbf5d0 [#8] Minor tweak to source_config 2013-02-15 12:50:09 +00:00
amercader 2ff5a11911 [#8] Use site user for harvesting actions
You don't need to create a 'harvest' sysadmin user any more.
By default this will be the internal site admin user. This is the
recommended setting, but if necessary it can be overridden by
the `ckanext.spatial.harvest.user_name` config option, eg to
support the old hardcoded 'harvest' user
2013-02-15 12:28:58 +00:00
amercader baf7b5da67 [#8] Rename harvested document model to ISODocument
GeminiDocument has been kept for backwards incompatibility.
2013-02-13 19:16:36 +00:00
amercader dadd174293 [#8] Make sure only list and dict extras are dumped as JSON 2013-02-13 18:48:53 +00:00
amercader 00a0b5946b [#8] Add transform_to_iso method
This can be overridden by custom harvesters willing to support non ISO
formats (like FGDC). It is called whrn the original_document and
original_format harvest object extras are present. Custom harvesters are
responsible for transforming the original document to ISO.
2013-02-13 18:33:58 +00:00
amercader 641ffad589 Remove silly debug message 2013-02-13 17:49:15 +00:00
amercader 305951aeb5 [#8] Make get_package_dict public and document
So it is more obvious that it can be overridden by custom harvesters.
2013-02-13 17:06:01 +00:00
amercader 1d8a4c17c4 [#8] Update harvesters for CsW, WAF and Doc sources
These are the new versions of the spatial harvesters with significant
improvement over previous ones.
2013-02-12 18:29:30 +00:00
amercader f153b0f4ba [#8] Minor fixes in base harvester 2013-02-12 18:26:03 +00:00
amercader c7d872af7e [#8] Rename source config object and method to avoid confusion 2013-02-12 18:07:05 +00:00
amercader 24270cb4cb [#8] Move Gemini harvesters and updated base harvester to own files
Prior to the merging of the new spatial harvesters, the existing ones
based on Gemini and UKLP have been moved to their own namespace
(ckanext.spatial.harvesters.gemini). The plugin points have been updated
so users currently using these harvesters will still be able to use them
as normal.

The base harvester (SpatialHarvester) has been updated with new methods,
most significally '_get_package_dict' and 'import_stage'. Note that
SpatialHarvester now extends HarvesterBase on ckanext-harvest, which had
some of its methods updated.

TODO: still some geo.data.gov specific bits!
2013-02-12 17:40:41 +00:00