b0780b2062
Fetch stage can also return "unchanged", same as the import stage. Used by DGU. It is useful to skip an object like this, to avoid saving the fetched content in a HarvestObject (saves disk usage).
2015-12-01 17:38:57 +00:00
414c33ac6b
Easy fix to stop test_queue.py using MockHarvester in test_queue2.py by mistake.
2015-12-01 16:55:42 +00:00
6c3efe53df
Fix job stats key - it is "errored" not "errors" - the keys are from report_status. And lots of PEP8.
2015-11-25 20:55:32 +00:00
e71cf35504
Fix queue tests
2015-11-20 14:25:31 +00:00
ede50aa3fb
Merge branch 'immediate-harvest'
2015-11-20 12:28:35 +00:00
3f42eb6ba0
Merge branch 'revision-id-fix'
...
Conflicts:
ckanext/harvest/tests/harvesters/test_ckanharvester.py
2015-11-20 12:28:17 +00:00
60c4371df4
Add "not modified" to the stats always returned.
2015-11-17 12:45:00 +00:00
ae7c500745
Merge branch 'master' into yhteentoimivuuspalvelut-job-reporting-fixes
2015-11-17 12:35:59 +00:00
8e02aedc65
Fix tests
2015-11-17 13:29:25 +01:00
c0a865e64e
Revert ok_ - makes it slightly less readable for little benefit.
2015-11-13 13:45:56 +00:00
42ab55cb6d
No longer need uuid since we clear db between tests now. Added ignore_missing because of occasional failures.
2015-11-13 13:32:55 +00:00
b150b50887
Move the SkipTest to include inherited tests too.
2015-11-13 12:44:27 +00:00
01a4bfd314
Patch test should skip if ckan version is wrong, rather than ignore all exceptions from posts of all tests. Remove FunctionalTestBaseWithoutClearBetweenTests now the tests are modernized.
2015-11-13 12:33:18 +00:00
0ce3748153
Do not use ActionError as this does not yet exist in CKAN 2.2
2015-11-13 12:01:19 +01:00
c33c6e8c13
Raise an error instead of falling back to harvest_source_update
...
As the behaviour of *_patch is clearly different from the *_update we
should raise an error if this action is called on a CKAN instance, where
the action is not available.
2015-11-13 11:41:53 +01:00
644fa49dd4
Make tests independent from cls.sysadmin
...
Generate unique harvest sources
2015-11-11 19:49:49 +01:00
ffca5cc3da
Use new factory style for fixtures
...
- Remove default_source_dict from tests
- Replace setup_class with constructor
- Create mixin for harvest source fixture
- Replace assert with assert_equal where appropriate
- Replace assert with ok_
- Remove dependency to global SOURCE_DICT
- General refactoring of tests
2015-11-11 19:07:59 +01:00
136fcb87d5
Make sure package_patch has a fallback for package_update on CKAN < 2.3
2015-11-11 11:37:23 +01:00
359da2eb69
Add test class for harvest_source_patch
2015-11-11 11:34:01 +01:00
679ed421e9
Merge branch 'master' of github.com:ckan/ckanext-harvest into immediate-harvest
...
Conflicts:
ckanext/harvest/logic/action/update.py
2015-11-04 09:44:05 +00:00
f0207ad38f
Merge branch 'master' of github.com:ckan/ckanext-harvest into yhteentoimivuuspalvelut-job-reporting-fixes
2015-11-04 09:36:55 +00:00
cbe9b40e66
Merge branch 'master' of github.com:ckan/ckanext-harvest into revision-id-fix
2015-11-04 09:36:08 +00:00
f9da3654f8
[ #184 ] Fix tests for older ckan versions.
2015-11-03 23:27:52 +00:00
5fba056c59
[ #184 ] Add tests
2015-11-03 23:19:05 +00:00
5a5260ff0b
Add test for harvest_source_clear since the PEP8 changes were quite a lot there.
2015-11-03 21:42:39 +00:00
91afc0e928
[ #178 ] Fix test for ckan 2.2, which does not suffer problem #180 .
2015-11-03 10:01:47 +00:00
f4f124c181
[ #178 ] Fix resouce_revision_id_fkey error. Fixes #178 .
2015-11-03 07:38:01 +00:00
c7fac36c1c
[ #107 ] "unchanged" response tested and related fixes
...
* fix "existing_package_dict" which wasn't containing metadata_modified (because of the schema in the context) so you never skipped an object.
* fix IntegrityError due to resource revision_id being harvested. No idea why this hasn't caused errors before now.
* "unchanged" is now checked in base instead of ckanharvester - makes sense. Looking at other harvesters, it's normal to return from the import_stage with the value returned from base._create_or_update_package so I've continued with that.
* "unchanged" response is now documented
* better report_status tests in test_queue2.
2015-11-03 00:22:53 +00:00
f1d2d5fdc4
[ #111 ] Run jobs straight away.
2015-10-28 21:58:36 +00:00
86630adab7
Merge branch 'include-exclude-org'
2015-10-27 15:52:55 +00:00
24574f485b
Setup harvest model in harvester tests
2015-10-23 15:43:01 +01:00
bc49149d5e
Merge branch 'master' into include-exclude-org
2015-10-23 14:36:53 +01:00
2f4adfb338
Merge branch 'tests'
2015-10-23 13:18:15 +01:00
eb9aa17862
Include/exclude orgs funcationality based on work by memaldi and ross.
2015-10-21 16:33:16 +00:00
f70c16bce7
Add framework for testing harvesters. Modernize existing tests.
2015-10-21 16:26:57 +00:00
82bdff2f34
Add tests
2015-10-01 17:59:17 +01:00
88d9ba0397
[ #136 ] Fix broken RabbitMQ queue names
...
The harvester command was still using the old ones.
Use specific ones for testing.
2015-06-11 13:56:22 +01:00
3e21ea4f82
Fix tests, set up Travis
...
TODO: sort out the tests properly, avoiding imports from the legacy ones
2015-04-07 13:31:45 +01:00
5da153c6b6
[ #65 ] harvest_object_create action
...
update to use schema and validators. Also accept more parameters to
data_dict.
2013-09-17 16:49:19 +01:00
1c36b33aaf
[ #59 ] Ignore auth when using site_user
2013-08-14 12:28:27 +01:00
4b6efd8ccb
Update queue tests
2013-04-22 17:55:46 +01:00
fdac761fba
Merge branch 'release-v2.0' into 2.0-dataset-sources
...
Conflicts:
ckanext/harvest/logic/action/get.py
ckanext/harvest/tests/test_queue.py
2012-12-20 16:16:30 +00:00
b940baacc0
make statistics use new report_field
2012-12-18 02:39:14 +00:00
6b42d96fe0
add report_status field
2012-12-17 23:50:26 +00:00
478326922b
Fix tests
...
* Adapt test_queue to harvest source datasets
* Don't use the same mock harvester on different datasets as it messes
the tests up
* Skip auth tests for the time being
2012-12-14 14:52:19 +00:00
c1b0415cb6
Merge branch 'release-v2.0' into 2.0-dataset-sources
...
Conflicts:
ckanext/harvest/model/__init__.py
2012-12-13 18:33:59 +00:00
0dde483992
Set job status to Finished when actually finishing it
...
Until now, harvest jobs were set to Finished just after sending all
objects to the fetch stage. Now every time the run command is run, jobs
are set to Running, and all previous Running jobs are checked to see if
all harvest objects have a state of Complete or Error. Only then the job
is flagged as Finished.
2012-12-13 18:19:22 +00:00
9d83322591
Fix config validator and add tests
2012-11-30 17:02:06 +00:00
803b228d1c
Update harvest source create and update logic functions
...
`harvest_source_create` and `harvest_source_update` now call
`package_create` and `package_update` respectively, making sure to
define a 'harvest_source' type. The returned dict uses the db_to_form
schema.
2012-11-30 14:11:24 +00:00
6db65b5826
made manual default not null
2012-11-05 13:17:32 +00:00
David Read
amercader
Stefan Oderbolz
joetsoi
kindly