ead9e67a33
updated def harvest_source_clear() to delete resource views, resource revisions & resources in CKAN >= 2.3
2015-02-23 17:02:21 +00:00
07e47acea3
Add a missing command to the install docs
2015-02-13 10:43:43 +00:00
b3ed6cae5a
Merge pull request #121 from metaodi/120-create-remote-orgs
...
Fetch remote organization via action api
2015-01-15 10:49:09 +00:00
c1bcee9684
Use str() to get the error message
2015-01-15 11:36:15 +01:00
191c39ce5c
Catch the more general URLError instead of HTTPError
...
HTTPError is a subclass of URLError, so catch URLError is enough. I
think the HTTP error code is not as important in this situation, so
catching the more generic error seems like the best solution.
2015-01-15 10:57:24 +01:00
b978c26e70
Use ContentFetchError instead of generic Exception
2015-01-15 00:49:11 +01:00
935b9dda01
Munge group name before fetching remote group
...
The API call /api/2/rest/package/<id> returns the display name of the
group instead of its ID. To properly match the group, munge the name
before calling /api/2/rest/group
2015-01-15 00:44:53 +01:00
ef35c21e2a
Improve exception handling with custom exception
...
1. Try whenever possible to catch specific exceptions
2. Raise custom exception where appropriate
3. Fix the exception handling in _get_group and _get_organization
2015-01-15 00:44:45 +01:00
a71cc495fb
Merge branch 'metaodi-117-url-already-in-use'
2015-01-14 11:16:26 +00:00
0fd38e0e54
Use _get_group as a fallback for remote orgs
...
First try to get a remote org from the remote Action API, if this fails
try to use the old rest api call, which works on older CKAN versions.
Only if both options fail, its currently not possible to get the remote
organization.
2015-01-14 00:10:27 +01:00
f214577872
Fetch remote organization via action api
...
Organizations used to be returned by /api/2/rest/group, this is what the
old implementation used to fetch the information to create the remote
organization on the local instance of CKAN.
With this commit the Action API is used to fetch the same information.
2015-01-13 14:46:53 +01:00
ea9debf714
Fix logic of conditional and make it more pythonic
2014-12-18 16:03:33 +01:00
08930d01bf
Make sure for new packages get a unique 'name'
2014-12-16 15:02:36 +01:00
82c7988bf3
Removed ResourceGroup from query when using CKAN 2.3 or above
2014-12-12 13:10:40 +00:00
a3affc9702
Fix validators on harvest_source_show schema
...
Remove validators on several keys so they don't get stripped during the
show validation.
2014-10-08 12:02:26 +01:00
623daa72c4
Merge branch 'waldvogel-clear-source-delete-related'
2014-09-29 13:50:59 +01:00
098b54f1e5
Merge branch 'clear-source-delete-related' of https://github.com/waldvogel/ckanext-harvest into waldvogel-clear-source-delete-related
2014-09-29 13:49:19 +01:00
e60e2eee03
Fix output for harvest_source_create/update
...
They were using an incorrect schema, so not returning a harvest source
like dict.
2014-09-29 12:43:37 +01:00
c9aff29613
Recommend Redis as the preferred backend
2014-09-19 11:38:38 +01:00
c9b4e10506
delete records from related and related_dataset when clearing source
2014-09-12 10:56:37 +02:00
1e0376cff6
fix typo
2014-09-10 10:33:13 +03:00
f6c1456abe
fix job reporting to have job finished timestamp when there was zero datasets to gather
2014-09-10 09:22:55 +03:00
97f09913cf
fix job reporting all datasets deleted when actually nothing changed during last two harvests
2014-09-10 09:22:44 +03:00
f34c1e6d74
Merge branch 'morty-99-all-non-ascii-tags'
2014-08-29 14:40:54 +01:00
8cf254f112
Merge branch '99-all-non-ascii-tags' of https://github.com/morty/ckanext-harvest into morty-99-all-non-ascii-tags
2014-08-29 14:40:43 +01:00
786e9011aa
Merge branch 'morty-101-modified-package-name'
2014-08-29 14:38:44 +01:00
546159744e
Merge branch '101-modified-package-name' of https://github.com/morty/ckanext-harvest into morty-101-modified-package-name
2014-08-29 14:38:33 +01:00
039ac7c0ad
Always remove harvest extras on after_show if there
...
Up until now we where relying on `for_edit` being present in the
context, but this is only added on the controllers. It's better to be
safe and remove them always. If needed (at index time) they will be
added afterwards.
2014-08-14 15:31:39 +01:00
8a2c072d4e
[101] Use name from database when reharvesting package
2014-08-12 11:18:48 +01:00
65cfade420
[99] Remove empty tags produced by munging all non-ascii tags
...
I thought this way of filtering was easier to read than filter(None, tags)
2014-08-07 17:05:16 +01:00
13dbb1eea4
Fix variable not defined
2014-07-30 15:49:02 +01:00
33000dbfbc
[ #88 ] Add redis to requirements
2014-06-27 16:59:15 +01:00
4f7563f066
Merge branch '91-reindex-list-fields-in-config'
2014-06-27 16:56:08 +01:00
58a873ac7a
[ #91 ] Remove config fields from source dict before indexing
...
We don't need them and will avoid indexing errors
2014-06-27 16:54:39 +01:00
a59ab4b5ff
[ #91 ] Consolidate all harvest source reindex code in a single action
...
Make it available to users with permissions on the harvest source
2014-06-27 16:48:14 +01:00
7459358fa1
Support for single import commands
...
We are now able to run `paster harvester import` for a single harvest
object or for a single dataset, providing ids or name.
2014-05-15 16:30:30 +01:00
d6998d8d04
Merge branch '96-harvest-object-encoding-errors'
2014-05-15 15:54:31 +01:00
2c6aaf5bb1
Merge branch 'master' into 96-harvest-object-encoding-errors
2014-05-15 15:52:13 +01:00
43f1d08255
[ #97 ] Persitent endpoint for datasets harvest objects
...
Contrary to `/harvest/object/xxx`, this endpoint is passed the dataset
id, thus it not depends on a particular object but the most recent one.
2014-04-30 17:45:07 +01:00
1b458b1772
[ #96 ] Handle encoding errors on harvest object endpoint
...
When parsing the harvest object content to see if it is an XML file,
etree.fromstring would fail id there are incorrect unicode errors.
2014-04-28 12:48:09 +01:00
040984d4ec
Merge branch 'rikdc-89-extras-check'
2014-02-28 18:23:43 +00:00
e3492b57e7
Update plugin.py
...
Updating plugin.py to check for the existence of the extras key in the data_dict.
2014-02-27 16:05:39 +00:00
d3cf5e58d1
[ #86 ] Fix duplicate extras
2014-02-11 18:16:49 +00:00
fbde0b8dc1
[ #87 ] Remove remote url_type from resources
...
Otherwise CKAN thinks they are uploads, datastore resources, etc, which
it can cause problems eg when displaying the URL of the resource. We
are just linking to the remote resource URL.
2014-02-11 17:27:19 +00:00
5739e541d7
[ #80 ] Support for Python 2.6 when handling xml exceptions
2014-02-10 18:44:46 +00:00
2a07a144fc
[ #84 ] Fix auth audit exception when creating datasets
...
This was caused by a combination of the auth audit leaking and the
harvester reusing the context for the package_show and package_create
actions. If the package is not found, package_show does not call
check_access, and the auth audit does not pass. This is stored in the
context (`__auth_audit`) and is raised next time that we call
get_action (when we call package_create with the same context)
It could potentially be fixed on master, but it is probably quite rare.
2014-02-10 18:22:48 +00:00
5b677b6099
[ #83 ] Fix key error when using default_groups
2014-02-10 13:16:58 +00:00
3693028009
Merge branch 'ogdch-79-optional-tag-munging'
2014-02-10 13:13:48 +00:00
bf11e4d330
Moved clean_tags check into _create_or_update_package method.
2014-02-10 09:29:01 +01:00
2ba9908653
Config option to munge tags changed to be consistent with other config options in this extension, and noted in README.
2014-01-29 10:55:51 +01:00
clementmouchet
Sean Hammond
David Read
Stefan Oderbolz
amercader
waldvogel
Jari Voutilainen
Tom Mortimer-Jones
Richard Claydon
Rachel Knowler