requires configuration-values:
ckan.harvest.status_mail.errored = true
email_to = %comma seperated list of emails to receive the email%
error_email_from = %sender of email%
ckan.site_title = %will be included in the email%
Fixes the print statements where they were doung to use print as a
function, and also to switch to .format() rather than %.
Also contains some flake8 whitespace changes as I suspect making this
PEP8ish will take several PRs.
CKAN 2.7.x upgrades to FontAwesome 4.x, so the CSS classes need to be
updated in order to work. But because this extension is used in older
CKAN versions as well, we simply add the new classes, while keeping the
old ones for backwards compatibility.
I sometimes see "connection timed out" message which are reported as
sqlalchemy.exc.DatabaseError, so by catching the latter exception, it'd
avoid the harvester to be stuck in "limbo" state.
As DatabaseError is a super-class of OperationalError, the latter would
still be catched.
logging.exception() already logs an ERROR message with exception
information, so there's no need to call both log.exception() and
log.error().
Along the way, make messages uniform in fetch_callback() and
gather_callback().
ckan.lib.base.redirect was removed in [1], guidance is to always use h.redirect_to instead. Manifested as 503 errors in the harvest UI against an install of CKAN master branch.
[1] 34f3f18e88
In order to get a quick overview over successful/failed harvesters, a
call to harvest_source_list with return_last_job_status=true can be used
to get this information.
By default return_last_job_status is False, and hence the extra
resources to grab this information is not wasted for every call, but
only if the client requests is explicitly.
The original 'status' field stays as-is, this PR introduces a new field
called 'last_job_status' to return this information.
The returned information is gather by a call to
harvest_source_status_show.
Previously purging the queue on the Redis backend would clear the whole
database, making it hard to share the same database with other parts of
CKAN. With this commit, only the keys that belong to ckanext-harvest and
the current CKAN instance are purged.
amercader
Stefanie Taepke
Ross Jones
seitenbau-govdata
Stefan Oderbolz
Alex Sadleir
Stefan Oderbolz
Knud Möller
etj
Denis Laxalde
Ubuntu
David Read
Mark Gregson
Jari Voutilainen
Raphael Stolt
Andy Gross
Florian Brucker
Jardel Weyrich