8ebb843052
Add documentation for clearsource command
2015-06-10 11:29:24 +02:00
61bc150ae6
Expose clear harvester source as a paster command
2015-06-10 11:19:10 +02:00
5fbfa2a4c1
i18n: add transifex setup
2015-06-10 10:07:07 +01:00
56e3ef4b79
i18n setup files
2015-06-10 09:55:36 +01:00
fd93a69da9
Document extra connection parameters
2015-06-01 17:55:37 +01:00
9f8aae3a18
Append site id to queue name
...
This allows multiple CKAN sites to share the same RabbitMQ exchange
(For the Redis backend this is handled via different Redis databases)
2015-06-01 17:54:22 +01:00
3e21ea4f82
Fix tests, set up Travis
...
TODO: sort out the tests properly, avoiding imports from the legacy ones
2015-04-07 13:31:45 +01:00
f72d6da521
Change toolkit import
...
Apparently on package installs this is not well supported
from ckan.plugins.toolkit import check_ckan_version
But this works:
from ckan.plugins import toolkit
toolkit.check_ckan_version(...
2015-03-19 12:48:46 +00:00
7a20e93716
Raise on startup import errors so we don't mask problems
...
Otherwise if there was eg an actual ImportError we jut got
2015-03-19 12:30:08,430 DEBUG [ckanext.harvest.plugin] No auth module
for action "update"
on the log
2015-03-19 12:48:15 +00:00
d6e9b80496
Merge pull request #118 from clementmouchet/114-remove_resource_groups
...
Removed ResourceGroup from query when using CKAN 2.3 or above
2015-02-24 09:56:44 +00:00
ead9e67a33
updated def harvest_source_clear() to delete resource views, resource revisions & resources in CKAN >= 2.3
2015-02-23 17:02:21 +00:00
07e47acea3
Add a missing command to the install docs
2015-02-13 10:43:43 +00:00
b3ed6cae5a
Merge pull request #121 from metaodi/120-create-remote-orgs
...
Fetch remote organization via action api
2015-01-15 10:49:09 +00:00
c1bcee9684
Use str() to get the error message
2015-01-15 11:36:15 +01:00
191c39ce5c
Catch the more general URLError instead of HTTPError
...
HTTPError is a subclass of URLError, so catch URLError is enough. I
think the HTTP error code is not as important in this situation, so
catching the more generic error seems like the best solution.
2015-01-15 10:57:24 +01:00
b978c26e70
Use ContentFetchError instead of generic Exception
2015-01-15 00:49:11 +01:00
935b9dda01
Munge group name before fetching remote group
...
The API call /api/2/rest/package/<id> returns the display name of the
group instead of its ID. To properly match the group, munge the name
before calling /api/2/rest/group
2015-01-15 00:44:53 +01:00
ef35c21e2a
Improve exception handling with custom exception
...
1. Try whenever possible to catch specific exceptions
2. Raise custom exception where appropriate
3. Fix the exception handling in _get_group and _get_organization
2015-01-15 00:44:45 +01:00
a71cc495fb
Merge branch 'metaodi-117-url-already-in-use'
2015-01-14 11:16:26 +00:00
0fd38e0e54
Use _get_group as a fallback for remote orgs
...
First try to get a remote org from the remote Action API, if this fails
try to use the old rest api call, which works on older CKAN versions.
Only if both options fail, its currently not possible to get the remote
organization.
2015-01-14 00:10:27 +01:00
f214577872
Fetch remote organization via action api
...
Organizations used to be returned by /api/2/rest/group, this is what the
old implementation used to fetch the information to create the remote
organization on the local instance of CKAN.
With this commit the Action API is used to fetch the same information.
2015-01-13 14:46:53 +01:00
ea9debf714
Fix logic of conditional and make it more pythonic
2014-12-18 16:03:33 +01:00
08930d01bf
Make sure for new packages get a unique 'name'
2014-12-16 15:02:36 +01:00
82c7988bf3
Removed ResourceGroup from query when using CKAN 2.3 or above
2014-12-12 13:10:40 +00:00
a3affc9702
Fix validators on harvest_source_show schema
...
Remove validators on several keys so they don't get stripped during the
show validation.
2014-10-08 12:02:26 +01:00
623daa72c4
Merge branch 'waldvogel-clear-source-delete-related'
2014-09-29 13:50:59 +01:00
098b54f1e5
Merge branch 'clear-source-delete-related' of https://github.com/waldvogel/ckanext-harvest into waldvogel-clear-source-delete-related
2014-09-29 13:49:19 +01:00
e60e2eee03
Fix output for harvest_source_create/update
...
They were using an incorrect schema, so not returning a harvest source
like dict.
2014-09-29 12:43:37 +01:00
c9aff29613
Recommend Redis as the preferred backend
2014-09-19 11:38:38 +01:00
c9b4e10506
delete records from related and related_dataset when clearing source
2014-09-12 10:56:37 +02:00
f34c1e6d74
Merge branch 'morty-99-all-non-ascii-tags'
2014-08-29 14:40:54 +01:00
8cf254f112
Merge branch '99-all-non-ascii-tags' of https://github.com/morty/ckanext-harvest into morty-99-all-non-ascii-tags
2014-08-29 14:40:43 +01:00
786e9011aa
Merge branch 'morty-101-modified-package-name'
2014-08-29 14:38:44 +01:00
546159744e
Merge branch '101-modified-package-name' of https://github.com/morty/ckanext-harvest into morty-101-modified-package-name
2014-08-29 14:38:33 +01:00
039ac7c0ad
Always remove harvest extras on after_show if there
...
Up until now we where relying on `for_edit` being present in the
context, but this is only added on the controllers. It's better to be
safe and remove them always. If needed (at index time) they will be
added afterwards.
2014-08-14 15:31:39 +01:00
8a2c072d4e
[101] Use name from database when reharvesting package
2014-08-12 11:18:48 +01:00
65cfade420
[99] Remove empty tags produced by munging all non-ascii tags
...
I thought this way of filtering was easier to read than filter(None, tags)
2014-08-07 17:05:16 +01:00
13dbb1eea4
Fix variable not defined
2014-07-30 15:49:02 +01:00
33000dbfbc
[ #88 ] Add redis to requirements
2014-06-27 16:59:15 +01:00
4f7563f066
Merge branch '91-reindex-list-fields-in-config'
2014-06-27 16:56:08 +01:00
58a873ac7a
[ #91 ] Remove config fields from source dict before indexing
...
We don't need them and will avoid indexing errors
2014-06-27 16:54:39 +01:00
a59ab4b5ff
[ #91 ] Consolidate all harvest source reindex code in a single action
...
Make it available to users with permissions on the harvest source
2014-06-27 16:48:14 +01:00
7459358fa1
Support for single import commands
...
We are now able to run `paster harvester import` for a single harvest
object or for a single dataset, providing ids or name.
2014-05-15 16:30:30 +01:00
d6998d8d04
Merge branch '96-harvest-object-encoding-errors'
2014-05-15 15:54:31 +01:00
2c6aaf5bb1
Merge branch 'master' into 96-harvest-object-encoding-errors
2014-05-15 15:52:13 +01:00
43f1d08255
[ #97 ] Persitent endpoint for datasets harvest objects
...
Contrary to `/harvest/object/xxx`, this endpoint is passed the dataset
id, thus it not depends on a particular object but the most recent one.
2014-04-30 17:45:07 +01:00
1b458b1772
[ #96 ] Handle encoding errors on harvest object endpoint
...
When parsing the harvest object content to see if it is an XML file,
etree.fromstring would fail id there are incorrect unicode errors.
2014-04-28 12:48:09 +01:00
040984d4ec
Merge branch 'rikdc-89-extras-check'
2014-02-28 18:23:43 +00:00
e3492b57e7
Update plugin.py
...
Updating plugin.py to check for the existence of the extras key in the data_dict.
2014-02-27 16:05:39 +00:00
d3cf5e58d1
[ #86 ] Fix duplicate extras
2014-02-11 18:16:49 +00:00
Stefan Oderbolz
joetsoi
amercader
David Read
clementmouchet
Sean Hammond
waldvogel
Tom Mortimer-Jones
Richard Claydon