Commit Graph

12 Commits

Author SHA1 Message Date
David Read 1a680f3fd3 [#158] Fix spaces encoding broken in previous merge. Tested with data.gov.uk. 2015-10-29 17:31:04 +00:00
David Read e2ab9e58e7 Merge remote-tracking branch 'origin/master' into 157-version-three-apify
Conflicts:
	ckanext/harvest/harvesters/ckanharvester.py
2015-10-28 14:34:27 +00:00
David Read 3f74c29c99 Merge branch 'master' into 157-version-three-apify 2015-10-27 17:45:27 +00:00
David Read 2a79873855 [#158] Use package search to get all datasets. Add paging search results. Store pkg_dict from search in the object rather than request it again in fetch_stage. 2015-10-27 17:33:22 +00:00
amercader 86630adab7 Merge branch 'include-exclude-org' 2015-10-27 15:52:55 +00:00
David Read b56fae8aed Fixes and tests
* Fix extras as a list of dicts
* Fix SOLR dates syntax - needed a Z
* Basic tests for this updated ckan harvester
* Now require CKAN 2.0 to be able to be able to save these packages in package_show form. Take advantage of this now we are such various imports from are definitely available, such as munge_tag.
* Add back compatibility for other harvesters supplying restful-like package_dicts to _create_or_update_package

TODO add back in the ability to harvest pre 2.0 CKANs with the RESTful calls (fallback or maybe configurable)
2015-10-23 17:30:28 +00:00
amercader 24574f485b Setup harvest model in harvester tests 2015-10-23 15:43:01 +01:00
David Read bc49149d5e Merge branch 'master' into include-exclude-org 2015-10-23 14:36:53 +01:00
amercader 2f4adfb338 Merge branch 'tests' 2015-10-23 13:18:15 +01:00
David Read eb9aa17862 Include/exclude orgs funcationality based on work by memaldi and ross. 2015-10-21 16:33:16 +00:00
David Read f70c16bce7 Add framework for testing harvesters. Modernize existing tests. 2015-10-21 16:26:57 +00:00
David Read 82bdff2f34 Add tests 2015-10-01 17:59:17 +01:00