CatalogueRevamping
/
harvester-d4science
14
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,274 Commits 3 Branches 0 Tags 2.5 MiB
Commit Graph

1274 Commits

Author SHA1 Message Date
Stefanie Taepke e1e6b44a44 extend TestHarvestErrorMail from FunctionalTestBase 
This prevents failing tests for older versions.
 2018-06-13 13:31:03 +02:00
Stefanie Taepke ad9e513c0d mock ckan.lib.mailer 2018-06-13 13:12:14 +02:00
Stefanie Taepke 00911a6924 test harvest-error-mail 2018-06-13 10:53:36 +02:00
Stefanie Taepke f965751bd5 send error-mail to sys-admins 2018-06-13 10:46:35 +02:00
Stefanie Taepke 20c9235270 use logic-actions instead of DB-Calls 2018-06-13 09:09:36 +02:00
Stefanie Taepke 5f414e2c99 add error-notifications in case of harvest-failure 
requires configuration-values:
ckan.harvest.status_mail.errored = true
email_to = %comma seperated list of emails to receive the email%
error_email_from = %sender of email%
ckan.site_title = %will be included in the email%
 2018-06-11 13:38:38 +02:00
Ross Jones fff9800266 Fix print statements to be Py3 friendly 
Fixes the print statements where they were doung to use print as a
function, and also to switch to .format() rather than %.

Also contains some flake8 whitespace changes as I suspect making this
PEP8ish will take several PRs.
 2018-06-08 09:53:13 +01:00
Stefan Oderbolz 2619185bf9
Merge pull request #318 from ckan/readme-test-tip-ckan-parameter 
Readme test tip ckan parameter
 2018-06-07 18:57:09 +02:00
Adrià Mercader 9efb9f4ee5
Merge pull request #327 from GovDataOfficial/config-option-for-dataset-name-append-type 
Add config option for dataset name append type
 2018-06-04 14:13:10 +02:00
seitenbau-govdata 431d202d4e Add tests and documentation for dataset name suffix config 2018-06-01 21:45:29 +02:00
seitenbau-govdata 938d0322f5 Add config option for dataset name append type 2018-06-01 21:43:37 +02:00
MinhChau 0f8d448864 feat: add groups filter config 2018-04-17 13:22:26 +09:00
David Read 47bdd576fb Minor formatting improvements - unquote 2018-04-03 12:24:33 +01:00
David Read a11c2a6697 Minor formatting improvements 2018-04-03 12:20:18 +01:00
David Read c2000e3804 Added a tip, based on running related tests in ckanext-dcat. 2018-04-03 12:13:03 +01:00
Stefan Oderbolz 65e3163015
Merge pull request #315 from maxious/fix-solr-erase-all 
Don't delete all solr documents/fail to index harvesters when harvest config blank
 2018-03-03 09:55:36 +01:00
Alex Sadleir c89ddc02a3 Don't delete all solr documents when harvest config blank 2018-03-03 19:19:54 +11:00
Adrià Mercader 7fa6ee8bc0
Merge pull request #313 from opendatazurich/compatibility-with-fa-4 
FontAwesome 4.x/CKAN 2.7.x compatibility
 2018-02-20 13:54:18 +01:00
Stefan Oderbolz 00fc2f8a8f Fix build 2018-02-20 09:14:20 +01:00
Stefan Oderbolz 2464dd27b3 Add the CSS classes for FontAwesome 4.x 
CKAN 2.7.x upgrades to FontAwesome 4.x, so the CSS classes need to be
updated in order to work. But because this extension is used in older
CKAN versions as well, we simply add the new classes, while keeping the
old ones for backwards compatibility.
 2018-02-13 16:26:29 +01:00
Stefan Oderbolz 0918a9a341
Add entry for last_error_free_job 2017-11-16 10:07:02 +01:00
Stefan Oderbolz f715aa5e12
Merge pull request #305 from knudmoeller/302-move-last-error-free-job-to-harvesterbase 
Move _last_error_free_job from CKANHarvester to HarvesterBase
 2017-11-16 10:04:15 +01:00
Knud Möller 717fdb35dd move _last_error_free_job from CKANHarvester to HarvesterBase 2017-11-10 12:19:25 +01:00
Stefan Oderbolz c313e076e8
Add clean_tags entry 2017-11-09 13:27:00 +01:00
Stefan Oderbolz 8910918674
Merge pull request #304 from geosolutions-it/303_clean_tags 
#303: fix clean_tags with tags dict
 2017-11-09 13:24:51 +01:00
etj d1dd4eb227 303: fix clean_tags with tags dict (fixes requested by review) 2017-11-08 13:46:44 +01:00
etj 41aa9c121f 303: fix clean_tags with tags dict 2017-11-07 14:14:50 +01:00
Stefan Oderbolz 2e2ca8e456 Release 1.1.0 2017-11-07 09:39:09 +01:00
Stefan Oderbolz 4b8a4f6683 Update CHANGELOG with recent changes/PRs 2017-11-07 09:34:23 +01:00
Stefan Oderbolz cccc469a06
Merge pull request #301 from dlax/db-errors 
Catch DatabaseError in fetch/gather callbacks
 2017-11-07 09:19:32 +01:00
Denis Laxalde 7bb9a2b5e4 Catch sqlalchemy's DatabaseError in fetch and gather callback 
I sometimes see "connection timed out" message which are reported as
sqlalchemy.exc.DatabaseError, so by catching the latter exception, it'd
avoid the harvester to be stuck in "limbo" state.

As DatabaseError is a super-class of OperationalError, the latter would
still be catched.
 2017-11-02 17:20:31 +01:00
Denis Laxalde cc44d03a41 Drop log.error() call redundant with prior log.exception() call 
logging.exception() already logs an ERROR message with exception
information, so there's no need to call both log.exception() and
log.error().

Along the way, make messages uniform in fetch_callback() and
gather_callback().
 2017-11-02 17:17:00 +01:00
Adrià Mercader c1661442c9
Merge pull request #298 from ckan/use-api-to-get-dataset-count 
Use the package_search API for the count of datasets
 2017-11-02 10:31:36 +00:00
Stefan Oderbolz 9bfd5866fb Use package_search to display package count of harvester 2017-11-01 08:24:32 +01:00
Stefan Oderbolz 9c86e8f73d Merge pull request #296 from aykutsimseks/add-abort-ui 
Abort stuck harvest jobs through UI
 2017-10-26 18:20:02 +02:00
Ubuntu 6714d4de58 Move Stop Button 2017-10-26 16:06:48 +00:00
Ubuntu 0d877bf768 Abort Jobs from UI 2017-10-26 14:42:39 +00:00
Adrià Mercader 685b5c48f2 Merge pull request #297 from ckan/fix-travis-build 
Fix travis build
 2017-10-26 11:57:48 +01:00
Stefan Oderbolz ac62951fe5 Fix travis build 
- Use correct version of CKAN
- install latest version of libcommons-fileupload-java
- Use the postgresql service of travis
 2017-10-25 17:20:55 +02:00
amercader b589797c47 Merge branch 'master' of github.com:ckan/ckanext-harvest 2017-07-27 16:09:19 +01:00
amercader 881a9581d5 Fix url_for depending tests 2017-07-27 16:09:00 +01:00
Adrià Mercader 4fde5afdd6 Merge pull request #290 from geosolutions-it/289-fix-default-extras 
[#289] Fix default_extras
 2017-07-21 10:14:26 +01:00
etj 207ac81d70 [289] Fix default_extras 2017-05-29 20:11:49 +02:00
Adrià Mercader de01277e50 Merge pull request #288 from ckan/explicit_exception_checks 
Minor test improvements
 2017-05-09 11:55:00 +01:00
David Read cc438786de More explicit checks of the exception thrown when checking harvest config. Also the default_groups test was checking the wrong thing completely. 2017-05-02 20:39:49 +00:00
Stefan Oderbolz bca526c34a Release 1.0.0 2017-03-30 23:50:39 +02:00
Stefan Oderbolz 291d573870 Add a "Releases" section to README 2017-03-30 23:50:39 +02:00
Stefan Oderbolz 834edfacce Merge pull request #286 from ckan/update-readme-contributing 
Update README to link to CONTRIBUTING
 2017-03-30 23:47:03 +02:00
Stefan Oderbolz a0562de442 Update README to link to CONTRIBUTING 
Add link to CKAN guidlines in CONTRIBUTING
 2017-03-30 23:25:25 +02:00
Stefan Oderbolz f72424a0b8 Merge pull request #285 from ckan/update-changelog 
Update CHANGELOG
 2017-03-30 23:15:20 +02:00