072698b4bb
Merge branch 'validator-raise'
2015-12-10 13:16:26 +00:00
4cc39e5ef4
Merge branch 'test-guid-fix'
2015-12-10 13:15:54 +00:00
04162ce9e4
Merge branch 'munge-tag'
2015-12-10 13:15:17 +00:00
4ca4b3a2f2
Merge pull request #213 from ckan/212-module-import-error
...
[#212 ] Fixes #212 - auth for harvest_job_create was broken.
2015-12-09 15:52:37 +00:00
8b8086fe48
[ #212 ] Fixes #212 - auth for harvest_job_create was broken.
2015-12-09 15:50:05 +00:00
84fb3e3325
Add support for the new ITranslation interface
...
Move the i18n files to the expected place, load the interface if CKAN >=
2.5.0
2015-12-09 14:46:12 +00:00
121e8bd918
Merge pull request #198 from ckan/catch-exceptions
...
Catch exceptions from urllib2.urlopen more comprehensively
2015-12-02 16:22:53 +00:00
414c33ac6b
Easy fix to stop test_queue.py using MockHarvester in test_queue2.py by mistake.
2015-12-01 16:55:42 +00:00
673a64a820
Merge pull request #200 from ckan/fix-error-report-status
...
Fix job stats key
2015-11-30 17:49:14 +00:00
154a3f42cd
Merge pull request #202 from metaodi/local-time-display
...
Fix name of class to enable automatic conversion of datetimes
2015-11-27 10:22:54 +00:00
798866d872
Fix name of class to enable automatic conversion of datetimes
...
when ckan/ckan#2505 got finally merged, the class name to mark datetimes
with a time part that can be automatically converted to the users
timezone, changed.
This commit makes sure this change is reflected in ckanext-harvest.
2015-11-26 01:49:11 +01:00
4fbaec0986
Improve harvester type error message. Add docstring.
2015-11-25 20:58:50 +00:00
6c3efe53df
Fix job stats key - it is "errored" not "errors" - the keys are from report_status. And lots of PEP8.
2015-11-25 20:55:32 +00:00
f0a2e9fb8e
CKAN revision API returns package ids not names (for v2 of the API). This ensures harvest guid is always the ID rather than the name.
2015-11-24 16:41:43 +00:00
392c13d828
If not revisions then we get a 404, so deal with that better.
2015-11-23 21:36:45 +00:00
4405066fab
Catch exceptions from urllib2.urlopen more comprehensively. I think 400 errors were from CKAN 0.6 or something like that - ignore now.
2015-11-23 21:26:32 +00:00
4b5014d381
Fix test for older ckan.
2015-11-23 18:27:04 +00:00
3b4daf0609
fix typo
2015-11-23 17:40:35 +00:00
bc26159fb6
tag_munge from ckan 2.2 fails the test with dashes, so use the harvest one for this ckan version.
2015-11-23 17:31:20 +00:00
52f7e0dd07
Use the ckan version of munge_tag if available, but provide a fallback for older ckans.
2015-11-23 12:48:05 +00:00
d1476e6f7a
Merge pull request #196 from metaodi/enable-custom-update-detection
...
Provide extension point to detect existing packages
2015-11-20 16:23:38 +00:00
129b1a0cf5
Enable custom solution to detect existing packages
...
With this change, all harvesters that extend the base harvester have the
possibility to use the very useful create_or_update method, but still
define their own way of detecting what package is the existing one.
This is very useful for harvest sources that have no knowledge of the
CKAN internal id, but have another way of finding previous package.
2015-11-20 16:31:47 +01:00
e71cf35504
Fix queue tests
2015-11-20 14:25:31 +00:00
f1ba2bcfb3
Namespace Redis keys to avoid conflicts between instances
...
The `ckan.site_id` config option (or `default` if missing) is used to
namespace the Redis keys: routing key and persistance key. Consumers
will only get the relevant keys for their instance.
2015-11-20 14:17:25 +00:00
920df684ae
Merge branch 'db-error'
2015-11-20 12:29:37 +00:00
ede50aa3fb
Merge branch 'immediate-harvest'
2015-11-20 12:28:35 +00:00
3f42eb6ba0
Merge branch 'revision-id-fix'
...
Conflicts:
ckanext/harvest/tests/harvesters/test_ckanharvester.py
2015-11-20 12:28:17 +00:00
1b0bb910a5
Merge branch '180-fix-modified-date'
...
Conflicts:
ckanext/harvest/harvesters/ckanharvester.py
ckanext/harvest/tests/harvesters/test_ckanharvester.py
2015-11-20 12:27:11 +00:00
36babfb26a
Merge branch 'yhteentoimivuuspalvelut-job-reporting-fixes'
2015-11-20 12:20:32 +00:00
60c4371df4
Add "not modified" to the stats always returned.
2015-11-17 12:45:00 +00:00
ae7c500745
Merge branch 'master' into yhteentoimivuuspalvelut-job-reporting-fixes
2015-11-17 12:35:59 +00:00
cc79eeb942
Merge pull request #192 from metaodi/consistent-api-response
...
Make sure all status keys are always returned
2015-11-17 12:34:27 +00:00
8e02aedc65
Fix tests
2015-11-17 13:29:25 +01:00
f9b87fff0c
Make sure all possible status are always returned
...
It makes it harded to parse the API response if you always have to check
if 'deleted' is set and if what value it has. I think to simply return 0
for all status value is good practice.
2015-11-17 11:43:11 +01:00
5f96346487
Merge pull request #191 from raphaelstolt/master
...
Catch JSONDecodeError when no JSON content
2015-11-16 11:42:32 +00:00
084723abb7
Catch JSONDecodeError when no JSON content
2015-11-16 10:59:18 +01:00
6415cdd740
Merge pull request #190 from ckan/metaodi-harvest-source-patch
...
Metaodi harvest source patch
2015-11-13 13:52:09 +00:00
c0a865e64e
Revert ok_ - makes it slightly less readable for little benefit.
2015-11-13 13:45:56 +00:00
42ab55cb6d
No longer need uuid since we clear db between tests now. Added ignore_missing because of occasional failures.
2015-11-13 13:32:55 +00:00
b150b50887
Move the SkipTest to include inherited tests too.
2015-11-13 12:44:27 +00:00
01a4bfd314
Patch test should skip if ckan version is wrong, rather than ignore all exceptions from posts of all tests. Remove FunctionalTestBaseWithoutClearBetweenTests now the tests are modernized.
2015-11-13 12:33:18 +00:00
1288a4d9e7
Reflow text to 79 char width. Warning not necessary with an exception I think.
2015-11-13 12:32:13 +00:00
0ce3748153
Do not use ActionError as this does not yet exist in CKAN 2.2
2015-11-13 12:01:19 +01:00
c33c6e8c13
Raise an error instead of falling back to harvest_source_update
...
As the behaviour of *_patch is clearly different from the *_update we
should raise an error if this action is called on a CKAN instance, where
the action is not available.
2015-11-13 11:41:53 +01:00
73aecb7f12
Add datastore_default user due to PR #2234 from ckan
2015-11-11 20:06:12 +01:00
644fa49dd4
Make tests independent from cls.sysadmin
...
Generate unique harvest sources
2015-11-11 19:49:49 +01:00
ffca5cc3da
Use new factory style for fixtures
...
- Remove default_source_dict from tests
- Replace setup_class with constructor
- Create mixin for harvest source fixture
- Replace assert with assert_equal where appropriate
- Replace assert with ok_
- Remove dependency to global SOURCE_DICT
- General refactoring of tests
2015-11-11 19:07:59 +01:00
136fcb87d5
Make sure package_patch has a fallback for package_update on CKAN < 2.3
2015-11-11 11:37:23 +01:00
359da2eb69
Add test class for harvest_source_patch
2015-11-11 11:34:01 +01:00
3f09010039
Add harvest_source_patch to API
2015-11-11 05:39:29 +01:00
amercader
David Read
Stefan Oderbolz
Raphael Stolt