Commit Graph

1104 Commits

Author SHA1 Message Date
amercader 3be2d6acd0 Update CHANGELOG 2019-10-03 11:52:40 +02:00
amercader 3793480a91 Fix Flake8 warnings 2019-10-03 11:50:15 +02:00
amercader 57718973ce Add exit-zero to the flake8 command for pep8 2019-10-03 11:39:24 +02:00
amercader 5ffe6d4bfa Resend awaiting objects in the DB that are not present in Redis
This happens when there is an ongoing harvest job and the keys on redis
are deleted for some reason (eg not persistent container)
2019-10-03 11:32:48 +02:00
amercader 5ad6d866ed Mark early errored job as finished
Otherwise it's displayed as Running forever
2019-10-03 11:22:23 +02:00
amercader 8e862c8f04 Fix wrong toolkit reference 2019-10-03 11:21:44 +02:00
Adrià Mercader 5c9c093803
Merge pull request #376 from ckan/remove-old-migrations
Remove old database migrations that targeted ancient versions
2019-10-02 15:58:00 +02:00
amercader 72591e83ad Add changelog notice 2019-10-01 12:08:29 +02:00
amercader 7ac70cd58e Remove old database migrations that targeted ancient versions
For this version it is safe to assume that is these migrations are no
longer needed. They could introduce hard to track errors as part of the
startup work they did.
2019-10-01 12:04:16 +02:00
amercader 819706ae5d Add further notes on extending ckan harvester 2019-08-26 13:48:09 +02:00
amercader 30f19713fa Add hook to extend the package dict in CKAN harvester 2019-08-26 13:38:30 +02:00
amercader 51bf79a626 Revert back to StrictRedis client 2019-08-26 12:26:05 +02:00
amercader 8b2f2f9a24 Merge branch 'default-to-ckan.redis.url' 2019-08-23 15:46:37 +02:00
Adrià Mercader 25775ffa7b
Merge pull request #358 from ckan/ckantoolkit
Use ckantoolkit to clean up imports
2019-08-23 14:39:54 +02:00
amercader b1b284a810 Default to the ckan.redis.url setting
99.9% of cases you probably want to use the same redis instance that
CKAN core uses so it makes sense to default to that setting. If it's not
set if will default to the hardcoded values (localhost:6379/0) which is
the same default as CKAN core. If there is an existing legacy setting
(ckan.harvest.mq.hostname) those will be used.
2019-08-23 13:34:17 +02:00
Stefan Oderbolz b8cf47bb91
Merge pull request #365 from usingsky/fix-SAWarning
Fix SAWarning when tracking update from ckan
2019-07-09 07:58:49 +02:00
Stefan Oderbolz 9eee08b034
Merge pull request #367 from GovDataOfficial/fix-broken-abort-job-in-ui
Fix abort job from ui
2019-07-01 17:16:20 +02:00
seitenbau-govdata b206c4cf36 Fix abort job from ui
* Fix abort job from ui which was broken by commit
ed288dca63
2019-07-01 12:19:36 +02:00
Jinil Lee 7e006e8c9e Fix SAWarning when tracking update from ckan
Warnings emitted when coercing full SQL fragments into text()
2019-06-01 18:08:11 +09:00
David Read 3a72337f1e
Merge pull request #359 from ckan/fix-model-codestyle
Code style changes to model and plugin
2019-03-18 19:55:31 +00:00
Stefan Oderbolz 5335d66316 Changelog entry for ckantoolkit 2019-03-18 17:48:25 +01:00
Stefan Oderbolz 8be627a454 Use ckantoolkit to avoid imports from new_test 2019-03-18 17:48:25 +01:00
Stefan Oderbolz 2c8a909c66
Add #357 to CHANGELOG 2019-03-18 14:22:46 +01:00
Stefan Oderbolz ecf578648b
Merge pull request #357 from alphagov/add-test-error-raise
Fix harvest request exceptions
2019-03-18 14:21:20 +01:00
Ken Tsang dfc147e6d7 Fixed codestyle in test_ckanharvester.py 2019-03-18 11:16:50 +00:00
Ken Tsang 95e6e934ac Add tests for raised errors during _get_content 2019-03-18 10:53:57 +00:00
Ken Tsang 1b609eeb60 import HTTPError to ckanharvester.py 2019-03-18 10:53:57 +00:00
Bruce Bolt 730b476c08 Fix harvest requests exceptions 2019-03-18 10:53:57 +00:00
Stefan Oderbolz 487e06fc24 Add flake8 config to setup.cfg
if the config is set, the output of `flake8` locally is correct without
    specifiying and further options.
2019-03-17 22:59:53 +01:00
Stefan Oderbolz ef3dc4a46a Update code style in model and plugin
This is based on the review feedback in #354
2019-03-17 22:59:15 +01:00
Stefan Oderbolz c00ed4f6e6
Update CHANGELOG 2019-03-17 22:43:24 +01:00
Stefan Oderbolz 9884214bc2
Merge pull request #354 from vrk-kpa/apply_flake8
Apply flake8
2019-03-17 22:41:41 +01:00
Stefan Oderbolz e89b3e31aa Changelog entry for #348 2019-03-17 22:26:15 +01:00
Jari Voutilainen 2fbac74c29 Fix importing assert_in 2019-03-06 14:12:27 +02:00
Jari Voutilainen 3aec5ed279 Fix importing, so that extensions will not break 2019-03-06 13:36:42 +02:00
Jari Voutilainen 749447de66 Fix typo 2019-03-06 13:19:48 +02:00
Jari Voutilainen 98364723d7 Apply flake8 2019-03-06 13:19:05 +02:00
Jari Voutilainen 618fa2a34b Add flake8 test to travis 2019-03-06 13:18:50 +02:00
Adrià Mercader 5aad13c2f9
Merge pull request #348 from alphagov/fix-migration
Add an 'owner_org' to the v3 package migration
2018-12-03 15:54:11 +01:00
Bruce Bolt 68bf4893c4 Add an 'owner_org' to the v3 package migration
The auto-migration fails because the new package does not have an
owner organisation ID assigned, so fails validation
2018-12-03 14:13:04 +00:00
Stefan Oderbolz 80b24dfac2
Changelog entry for #344 2018-11-12 09:42:32 +01:00
Stefan Oderbolz e66362d113
Merge pull request #344 from alphagov/tls-patches
Fix SSL problems for old versions of Python 2.7.x
2018-11-12 09:17:28 +01:00
Andrea Grandi 5b84067790
Remove not needed dependencies 2018-11-05 10:35:28 +00:00
Stefan Oderbolz 7ea25dfbf5
Merge pull request #345 from ckan/update-changelog
Add CHANGELOG entries for 1.1.x releases
2018-11-01 19:03:30 +01:00
Stefan Oderbolz 7f32be1c27 Bump to 1.1.4 2018-11-01 17:01:00 +01:00
Stefan Oderbolz df3c124739 Add CHANGELOG entry for #338 2018-11-01 16:58:36 +01:00
Stefan Oderbolz 443a6062e0 Add CHANGELOG entries for 1.1.x releases 2018-11-01 16:58:29 +01:00
Andrea Grandi 685843e583
Restore missing import 2018-11-01 14:00:38 +00:00
Andrea Grandi 6f82f19049
Simplify exception handling 2018-10-30 09:52:38 +00:00
Stefan Oderbolz c21c162799
Merge pull request #338 from brucebolt/fix-attribute-error
A harvester may not have an info method
2018-10-29 16:08:13 +01:00