443d690ac8
Fixed big typo error.
2015-11-02 16:45:16 +00:00
53f692b802
Merge remote-tracking branch 'remotes/upstream/master'
2015-11-02 16:00:14 +00:00
1702cf2f09
Remove ', None' on .get() calls because it's the default value.
2015-11-02 15:51:25 +00:00
0c19acba78
Changed double quotes to single quotes in docstrings.
2015-11-02 15:50:04 +00:00
a6069d93db
Fixed bug where the harvest source url validator would validate against
...
all harvest sources that were ever created instead of just sources that
were currently enabled.
2015-10-30 16:59:04 +00:00
3f37ae5f45
Corrected docstring.
2015-10-30 16:11:25 +00:00
02b81187df
Fixed bug with deleting harvest source's which have a custom
...
configuration. Added PEP-8 compliance.
2015-10-30 15:15:41 +00:00
55325f5940
Updated harvest source url validator to allow for duplicate URL's with
...
unique configs.
2015-10-30 11:59:24 +00:00
2c41293c9c
Updated the validator to check for unique sets as well as URL.
2015-10-29 18:30:51 +00:00
39ce744368
Modified to make PEP-8 compliant.
2015-10-29 17:18:51 +00:00
421e6da660
Add run_test, job_abort, source commands
...
* run_test - for running a whole harvest on the command-line
* job_abort - for aborting a limbo job
* source - for showing a single harvest source
* allowing a source to be specified by name in several commands
2015-10-28 17:51:58 +00:00
86630adab7
Merge branch 'include-exclude-org'
2015-10-27 15:52:55 +00:00
24574f485b
Setup harvest model in harvester tests
2015-10-23 15:43:01 +01:00
bc49149d5e
Merge branch 'master' into include-exclude-org
2015-10-23 14:36:53 +01:00
2f4adfb338
Merge branch 'tests'
2015-10-23 13:18:15 +01:00
3c6cc55be0
Only flush keys on the current Redis database
2015-10-23 11:52:22 +01:00
fdbade465f
Merge branch 'master' into purge
2015-10-23 11:33:43 +01:00
d950b13400
Merge branch 'unique-names-improved'
2015-10-23 11:02:49 +01:00
501edffe2d
Merge branch 'master' into migration-states
2015-10-23 10:59:04 +01:00
3e4a9933ce
Remove prints.
2015-10-21 16:52:19 +00:00
dc7af5d150
Remove prints.
2015-10-21 16:38:03 +00:00
eb9aa17862
Include/exclude orgs funcationality based on work by memaldi and ross.
2015-10-21 16:33:16 +00:00
f70c16bce7
Add framework for testing harvesters. Modernize existing tests.
2015-10-21 16:26:57 +00:00
d1f84295f8
purge_queues command now has warning about impact of Redis flushall, plus add some (log) output when you run a purge.
2015-10-21 16:12:40 +00:00
6360681a8f
[ #105 ] Fix order of deletes, as agreed with @florianm.
2015-10-12 15:57:27 +01:00
82bdff2f34
Add tests
2015-10-01 17:59:17 +01:00
be3e88086a
Generating unique names improved
...
* Harvesters that change the name when the title changes have had a
problem when the change is small and a number was unnecessarily
appended. e.g. "Trees "->"Trees" meant _gen_new_name("Trees") returned
"trees1". Now you can specify the existing value and it will return
that if it still holds.
* Maximum dataset name length is now adhered to.
* To make a name unique, a sequential number is now added, since for
users that is more understandable and pleasant. However hex digits are
still an option, for those that want to harvest concurrently.
2015-10-01 17:53:03 +01:00
a6cdda0a14
set max version to 2.4.99
2015-08-19 08:41:42 +00:00
1905caa961
upgrade harvest_source_clear to not delete from authz models removed in migration 078
2015-08-19 10:25:20 +08:00
46f7b32b04
Merge branch 'master' of github.com:okfn/ckanext-harvest into migration-states
2015-07-22 10:13:55 +01:00
2da918c2e4
Fix migration for old harvests so that ones that errored are correctly marked. Added helpful comments in model.
2015-07-22 10:13:02 +01:00
ab76830e85
[ #145 ] Throw + catch a custom exception if there are no jobs to run
...
If there are no harvesting jobs to run, there was always an ugly
exception message when using the paster command. This replaces the ugly
output with a proper message and uses a custom exception to allow others
to deal with this error differently.
2015-07-20 18:41:50 +02:00
83dd0b4b68
[ #138 ] Add data attributes to support timezone conversion
2015-07-09 22:35:54 +02:00
4dc2f7367d
[ #139 ] Delete package relationships when clearing a harvest source
2015-06-26 17:20:23 +02:00
88d9ba0397
[ #136 ] Fix broken RabbitMQ queue names
...
The harvester command was still using the old ones.
Use specific ones for testing.
2015-06-11 13:56:22 +01:00
673dfc9882
[ #127 ] Use site user on the CKAN harvester
...
Add missing call
2015-06-11 10:38:33 +01:00
d3a3f09ad1
[ #127 ] Use site user on the CKAN harvester
...
To avoid having to create a 'harvest' sysadmin explicitly. It will still
be used if present, but if not the site user will be used. You can also
define to user to use via a config option.
2015-06-11 10:19:07 +01:00
b17c3269b5
Merge branch 'clear-command' of https://github.com/metaodi/ckanext-harvest into metaodi-clear-command
2015-06-10 15:32:37 +01:00
64ff0f3a3a
Use single quotes to be consistent
2015-06-10 16:22:04 +02:00
2a2d85f60c
Wording changes for clearsource and rmsource
2015-06-10 16:19:23 +02:00
92b93c53fc
add some translation strings
2015-06-10 12:14:20 +01:00
8ebb843052
Add documentation for clearsource command
2015-06-10 11:29:24 +02:00
61bc150ae6
Expose clear harvester source as a paster command
2015-06-10 11:19:10 +02:00
9f8aae3a18
Append site id to queue name
...
This allows multiple CKAN sites to share the same RabbitMQ exchange
(For the Redis backend this is handled via different Redis databases)
2015-06-01 17:54:22 +01:00
3e21ea4f82
Fix tests, set up Travis
...
TODO: sort out the tests properly, avoiding imports from the legacy ones
2015-04-07 13:31:45 +01:00
f72d6da521
Change toolkit import
...
Apparently on package installs this is not well supported
from ckan.plugins.toolkit import check_ckan_version
But this works:
from ckan.plugins import toolkit
toolkit.check_ckan_version(...
2015-03-19 12:48:46 +00:00
7a20e93716
Raise on startup import errors so we don't mask problems
...
Otherwise if there was eg an actual ImportError we jut got
2015-03-19 12:30:08,430 DEBUG [ckanext.harvest.plugin] No auth module
for action "update"
on the log
2015-03-19 12:48:15 +00:00
d6e9b80496
Merge pull request #118 from clementmouchet/114-remove_resource_groups
...
Removed ResourceGroup from query when using CKAN 2.3 or above
2015-02-24 09:56:44 +00:00
ead9e67a33
updated def harvest_source_clear() to delete resource views, resource revisions & resources in CKAN >= 2.3
2015-02-23 17:02:21 +00:00
b3ed6cae5a
Merge pull request #121 from metaodi/120-create-remote-orgs
...
Fetch remote organization via action api
2015-01-15 10:49:09 +00:00
Mark Winterbottom
David Read
amercader
Florian Mayer
Stefan Oderbolz
joetsoi
clementmouchet