CatalogueRevamping
/
harvester-d4science
14
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,087 Commits 3 Branches 0 Tags 2.5 MiB
Commit Graph

1087 Commits

Author SHA1 Message Date
Adrià Mercader fd5f2deaf7
Merge pull request #335 from dennisrpnw/master 
Ensures the AND operator for fq in solr
 2018-10-25 12:14:13 +02:00
Bruce Bolt e7384106c3 A harvester may not have an info method 
This fixes the error `AttributeError: 'HarvesterBase' object has
no attribute 'info'` which prevents new harvest sources from
being created.
 2018-10-25 10:32:50 +01:00
Dennis Rupnow 1a63250e4b Ensures the AND operator for fq in solr 2018-08-30 16:33:55 +02:00
Gavin Cannizzaro 9f8b98e54f Use Redis password from configuration when present. 2018-07-12 10:14:24 -04:00
Adrià Mercader dcbc5d5d05
Merge pull request #329 from stefina/send-harvest-error-mail-to-members 
Send harvest-error-mails to organization-admins
 2018-06-15 11:31:45 +02:00
Stefanie Taepke cd6f75251f test if error-mails are only sent to respective org’s admins 2018-06-14 14:45:26 +02:00
Stefanie Taepke 304bbcec06 send harvest-error-mails to organization-admins 2018-06-14 11:07:49 +02:00
amercader 7dd82a0e01 Bump to 1.1.1 2018-06-13 15:31:32 +02:00
amercader 300da315e0 Merge branch 'stefina-error-mail' 2018-06-13 14:22:16 +02:00
amercader 9085533909 Readme tweaks 2018-06-13 14:19:30 +02:00
amercader a6cb710127 Merge branch 'error-mail' of https://github.com/stefina/ckanext-harvest into stefina-error-mail 2018-06-13 14:17:42 +02:00
Adrià Mercader 818c76dd97
Merge pull request #328 from servercode/py3-proof-print 
Fix print statements to be Py3 friendly
 2018-06-13 14:00:01 +02:00
Stefanie Taepke c29b52867c update readme for receivers of error-mails 2018-06-13 13:54:13 +02:00
Stefanie Taepke e1e6b44a44 extend TestHarvestErrorMail from FunctionalTestBase 
This prevents failing tests for older versions.
 2018-06-13 13:31:03 +02:00
Stefanie Taepke ad9e513c0d mock ckan.lib.mailer 2018-06-13 13:12:14 +02:00
Stefanie Taepke 00911a6924 test harvest-error-mail 2018-06-13 10:53:36 +02:00
Stefanie Taepke f965751bd5 send error-mail to sys-admins 2018-06-13 10:46:35 +02:00
Stefanie Taepke 20c9235270 use logic-actions instead of DB-Calls 2018-06-13 09:09:36 +02:00
Stefanie Taepke 5f414e2c99 add error-notifications in case of harvest-failure 
requires configuration-values:
ckan.harvest.status_mail.errored = true
email_to = %comma seperated list of emails to receive the email%
error_email_from = %sender of email%
ckan.site_title = %will be included in the email%
 2018-06-11 13:38:38 +02:00
Ross Jones fff9800266 Fix print statements to be Py3 friendly 
Fixes the print statements where they were doung to use print as a
function, and also to switch to .format() rather than %.

Also contains some flake8 whitespace changes as I suspect making this
PEP8ish will take several PRs.
 2018-06-08 09:53:13 +01:00
Stefan Oderbolz 2619185bf9
Merge pull request #318 from ckan/readme-test-tip-ckan-parameter 
Readme test tip ckan parameter
 2018-06-07 18:57:09 +02:00
Adrià Mercader 9efb9f4ee5
Merge pull request #327 from GovDataOfficial/config-option-for-dataset-name-append-type 
Add config option for dataset name append type
 2018-06-04 14:13:10 +02:00
seitenbau-govdata 431d202d4e Add tests and documentation for dataset name suffix config 2018-06-01 21:45:29 +02:00
seitenbau-govdata 938d0322f5 Add config option for dataset name append type 2018-06-01 21:43:37 +02:00
MinhChau 0f8d448864 feat: add groups filter config 2018-04-17 13:22:26 +09:00
David Read 47bdd576fb Minor formatting improvements - unquote 2018-04-03 12:24:33 +01:00
David Read a11c2a6697 Minor formatting improvements 2018-04-03 12:20:18 +01:00
David Read c2000e3804 Added a tip, based on running related tests in ckanext-dcat. 2018-04-03 12:13:03 +01:00
Stefan Oderbolz 65e3163015
Merge pull request #315 from maxious/fix-solr-erase-all 
Don't delete all solr documents/fail to index harvesters when harvest config blank
 2018-03-03 09:55:36 +01:00
Alex Sadleir c89ddc02a3 Don't delete all solr documents when harvest config blank 2018-03-03 19:19:54 +11:00
Adrià Mercader 7fa6ee8bc0
Merge pull request #313 from opendatazurich/compatibility-with-fa-4 
FontAwesome 4.x/CKAN 2.7.x compatibility
 2018-02-20 13:54:18 +01:00
Stefan Oderbolz 00fc2f8a8f Fix build 2018-02-20 09:14:20 +01:00
Stefan Oderbolz 2464dd27b3 Add the CSS classes for FontAwesome 4.x 
CKAN 2.7.x upgrades to FontAwesome 4.x, so the CSS classes need to be
updated in order to work. But because this extension is used in older
CKAN versions as well, we simply add the new classes, while keeping the
old ones for backwards compatibility.
 2018-02-13 16:26:29 +01:00
Stefan Oderbolz 0918a9a341
Add entry for last_error_free_job 2017-11-16 10:07:02 +01:00
Stefan Oderbolz f715aa5e12
Merge pull request #305 from knudmoeller/302-move-last-error-free-job-to-harvesterbase 
Move _last_error_free_job from CKANHarvester to HarvesterBase
 2017-11-16 10:04:15 +01:00
Knud Möller 717fdb35dd move _last_error_free_job from CKANHarvester to HarvesterBase 2017-11-10 12:19:25 +01:00
Stefan Oderbolz c313e076e8
Add clean_tags entry 2017-11-09 13:27:00 +01:00
Stefan Oderbolz 8910918674
Merge pull request #304 from geosolutions-it/303_clean_tags 
#303: fix clean_tags with tags dict
 2017-11-09 13:24:51 +01:00
etj d1dd4eb227 303: fix clean_tags with tags dict (fixes requested by review) 2017-11-08 13:46:44 +01:00
etj 41aa9c121f 303: fix clean_tags with tags dict 2017-11-07 14:14:50 +01:00
Stefan Oderbolz 2e2ca8e456 Release 1.1.0 2017-11-07 09:39:09 +01:00
Stefan Oderbolz 4b8a4f6683 Update CHANGELOG with recent changes/PRs 2017-11-07 09:34:23 +01:00
Stefan Oderbolz cccc469a06
Merge pull request #301 from dlax/db-errors 
Catch DatabaseError in fetch/gather callbacks
 2017-11-07 09:19:32 +01:00
Denis Laxalde 7bb9a2b5e4 Catch sqlalchemy's DatabaseError in fetch and gather callback 
I sometimes see "connection timed out" message which are reported as
sqlalchemy.exc.DatabaseError, so by catching the latter exception, it'd
avoid the harvester to be stuck in "limbo" state.

As DatabaseError is a super-class of OperationalError, the latter would
still be catched.
 2017-11-02 17:20:31 +01:00
Denis Laxalde cc44d03a41 Drop log.error() call redundant with prior log.exception() call 
logging.exception() already logs an ERROR message with exception
information, so there's no need to call both log.exception() and
log.error().

Along the way, make messages uniform in fetch_callback() and
gather_callback().
 2017-11-02 17:17:00 +01:00
Adrià Mercader c1661442c9
Merge pull request #298 from ckan/use-api-to-get-dataset-count 
Use the package_search API for the count of datasets
 2017-11-02 10:31:36 +00:00
Stefan Oderbolz 9bfd5866fb Use package_search to display package count of harvester 2017-11-01 08:24:32 +01:00
Stefan Oderbolz 9c86e8f73d Merge pull request #296 from aykutsimseks/add-abort-ui 
Abort stuck harvest jobs through UI
 2017-10-26 18:20:02 +02:00
Ubuntu 6714d4de58 Move Stop Button 2017-10-26 16:06:48 +00:00
Ubuntu 0d877bf768 Abort Jobs from UI 2017-10-26 14:42:39 +00:00