CatalogueRevamping
/
harvester-d4science
14
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,090 Commits 3 Branches 0 Tags 2.5 MiB
Commit Graph

1090 Commits

Author SHA1 Message Date
Ken Tsang b625f642dc Add force-import to reharvest datasets 
- datasets that were created incorrectly because of bugs in the system would not otherwise be reharvested  unless the harvest source changed.
 2019-07-24 09:49:43 +01:00
kentsanggds 5671bdf932
Merge pull request #6 from alphagov/comment-out-harvest_source-stats 
Comment out harvest_source statistics
 2019-06-14 15:14:15 +01:00
Ken Tsang 937e27d24c Comment out harvest_source statistics 2019-06-14 13:29:53 +01:00
kentsanggds 4071faf221
Merge pull request #5 from alphagov/handle-redis-errors 
Add try except around Redis set to handle corrupt redis data
 2019-05-09 10:08:45 +01:00
Ken Tsang 0cbb3579a9 Update Redis exception to drop stack trace 
Full stack trace is probably not needed as the Redis data corruption is probably happening somewhere else, the error log should make it easier to investigate it when it does happen.
 2019-05-08 11:26:05 +01:00
Ken Tsang edcf80c944 Add try except around Redis set to handle corrupt redis data 
- prevents bad data from stopping harvest processing
- log the error for investigation, full stack trace output as part of log
 2019-05-03 13:13:31 +01:00
Edward Kerry fb0cf32546
Merge pull request #4 from alphagov/add-not-modified-status 
Add `Not modified` statistic report for unchanged imports.
 2019-04-15 13:17:28 +01:00
Ken Tsang 4772bb08b7 Don't send the job to the gather queue when running the paster run_test command 2019-04-12 11:32:45 +01:00
Ken Tsang 93a839efce Use == rather than `is` for string comparison 2019-04-12 11:31:04 +01:00
Ken Tsang a9e32e755a Add not modified status to job details and list 2019-04-12 11:30:31 +01:00
Stefan Oderbolz 2c8a909c66
Add #357 to CHANGELOG 2019-03-18 14:22:46 +01:00
Stefan Oderbolz ecf578648b
Merge pull request #357 from alphagov/add-test-error-raise 
Fix harvest request exceptions
 2019-03-18 14:21:20 +01:00
Ken Tsang dfc147e6d7 Fixed codestyle in test_ckanharvester.py 2019-03-18 11:16:50 +00:00
Ken Tsang 95e6e934ac Add tests for raised errors during _get_content 2019-03-18 10:53:57 +00:00
Ken Tsang 1b609eeb60 import HTTPError to ckanharvester.py 2019-03-18 10:53:57 +00:00
Bruce Bolt 730b476c08 Fix harvest requests exceptions 2019-03-18 10:53:57 +00:00
Stefan Oderbolz c00ed4f6e6
Update CHANGELOG 2019-03-17 22:43:24 +01:00
Stefan Oderbolz 9884214bc2
Merge pull request #354 from vrk-kpa/apply_flake8 
Apply flake8
 2019-03-17 22:41:41 +01:00
Stefan Oderbolz e89b3e31aa Changelog entry for #348 2019-03-17 22:26:15 +01:00
Jari Voutilainen 2fbac74c29 Fix importing assert_in 2019-03-06 14:12:27 +02:00
Jari Voutilainen 3aec5ed279 Fix importing, so that extensions will not break 2019-03-06 13:36:42 +02:00
Jari Voutilainen 749447de66 Fix typo 2019-03-06 13:19:48 +02:00
Jari Voutilainen 98364723d7 Apply flake8 2019-03-06 13:19:05 +02:00
Jari Voutilainen 618fa2a34b Add flake8 test to travis 2019-03-06 13:18:50 +02:00
Adrià Mercader 5aad13c2f9
Merge pull request #348 from alphagov/fix-migration 
Add an 'owner_org' to the v3 package migration
 2018-12-03 15:54:11 +01:00
Bruce Bolt 68bf4893c4 Add an 'owner_org' to the v3 package migration 
The auto-migration fails because the new package does not have an
owner organisation ID assigned, so fails validation
 2018-12-03 14:13:04 +00:00
Stefan Oderbolz 80b24dfac2
Changelog entry for #344 2018-11-12 09:42:32 +01:00
Stefan Oderbolz e66362d113
Merge pull request #344 from alphagov/tls-patches 
Fix SSL problems for old versions of Python 2.7.x
 2018-11-12 09:17:28 +01:00
Andrea Grandi 5b84067790
Remove not needed dependencies 2018-11-05 10:35:28 +00:00
Stefan Oderbolz 7ea25dfbf5
Merge pull request #345 from ckan/update-changelog 
Add CHANGELOG entries for 1.1.x releases
 2018-11-01 19:03:30 +01:00
Stefan Oderbolz 7f32be1c27 Bump to 1.1.4 2018-11-01 17:01:00 +01:00
Stefan Oderbolz df3c124739 Add CHANGELOG entry for #338 2018-11-01 16:58:36 +01:00
Stefan Oderbolz 443a6062e0 Add CHANGELOG entries for 1.1.x releases 2018-11-01 16:58:29 +01:00
Andrea Grandi 685843e583
Restore missing import 2018-11-01 14:00:38 +00:00
Andrea Grandi 6f82f19049
Simplify exception handling 2018-10-30 09:52:38 +00:00
Stefan Oderbolz c21c162799
Merge pull request #338 from brucebolt/fix-attribute-error 
A harvester may not have an info method
 2018-10-29 16:08:13 +01:00
Andrea Grandi 6b6458f2ea
Use requests lib instead of urllib2 2018-10-29 11:50:39 +00:00
amercader 204b1f301a One more fix for 2.9 2018-10-26 14:11:20 +02:00
amercader 128fddffc3 Bump to 1.1.3 2018-10-26 13:55:44 +02:00
amercader da9232348c Merge branch '2.9-fixes' 2018-10-26 13:55:23 +02:00
amercader ed288dca63 Reduce usage of c vars, add more tests 2018-10-26 13:38:09 +02:00
amercader 7d1bcdf8bd Merge branch '2.9-support' 2018-10-25 15:50:39 +02:00
amercader fa0e277324 Bump to 1.1.2 2018-10-25 15:49:38 +02:00
amercader dea0bd09ad Remove unused controller 2018-10-25 15:48:42 +02:00
amercader cb5186962e Support for CKAN 2.9 
Use objects from toolkit, remove old checks using missing c properties,
Add legacy routes
 2018-10-25 15:39:57 +02:00
amercader 7a22a3cf46 Test CKAN 2.8 2018-10-25 14:58:37 +02:00
amercader c72142cdf3 Merge branch 'etri-odp-master' 2018-10-25 13:05:09 +02:00
amercader 7bad691415 Add tests and docs for include/exclude groups 2018-10-25 13:04:37 +02:00
amercader 753d29d5ff Merge branch 'master' of https://github.com/etri-odp/ckanext-harvest into etri-odp-master 2018-10-25 12:47:26 +02:00
amercader 3e5f98e708 Document redis password param 2018-10-25 12:20:47 +02:00