Some improvements on the endpoints that return the contents of the
harvest objects:
* Nicer URLs with redirects to the old ones
* Returning the raw harvest object content is available on the main
harvest extension, so just redirect there
* Support for showing the original document of a harvest object, if
present
* Suport for defining a custom XSLT for the HTML view, via
ckanext.spatial.harvest.xslt_html_content
ckanext.spatial.harvest.xslt_html_content_original
Prior to the merging of the new spatial harvesters, the existing ones
based on Gemini and UKLP have been moved to their own namespace
(ckanext.spatial.harvesters.gemini). The plugin points have been updated
so users currently using these harvesters will still be able to use them
as normal.
The base harvester (SpatialHarvester) has been updated with new methods,
most significally '_get_package_dict' and 'import_stage'. Note that
SpatialHarvester now extends HarvesterBase on ckanext-harvest, which had
some of its methods updated.
TODO: still some geo.data.gov specific bits!
Add new mandatory fields when creating sources, status dict has new
keys, CKAN lower cases formats, take into account harvest source
datasets.
Added a local getcapabilities response to avoid remote 404s.
Note that the TestValidation tests need to be fixed, as 27c4ee81e
removed the validation from the gather stage.
- Added a config option ('ckanext.spatial.use_postgis_sorting') to
activate this as this behaviour will be deprecated in the future
in favour of Solr 4 spatial sorting capabilities.
Also fixed the tests
Conflicts:
ckanext/spatial/plugin.py
Reverting changes for "#276 Coupled Resource" on master as it is INSPIRE-specific. Moving to datagovuk/ckanext-spatial branch dgu.
This reverts commit 91e547a622.
Reverting changes for "#276 Coupled Resource" on master as it is INSPIRE-specific. Moving to datagovuk/ckanext-spatial branch dgu.
This reverts commit 01536873b9.
Conflicts:
ckanext/spatial/harvesters.py
To make easier to filter and display errors on the UI, the validators
have been modified to return the message and line number separately. The
return format for validators is now:
(is_valid, [(error_message_string, error_line_number)])
Also the XSD based validators were returning only the last error found on
the document, instead of iterating the whole error log. Harvesters should
create a harvest object error for each of this validation errors.
Tests have been adapted to these changes.
Tests are passing, apart from a couple which didn't work before:
* test_functional.py -> functional/test_package.py (3 failures in 4)
* functional/test_dataset_map.py (1 fail in 1)
There may be some code errors still untested.
Renamed Validator -> Validators to make more sense.
fxia
amercader
David Read