CatalogueRevamping
/
harvester-d4science
14
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,104 Commits 3 Branches 0 Tags 2.5 MiB
Commit Graph

125 Commits

Author SHA1 Message Date
amercader 819706ae5d Add further notes on extending ckan harvester 2019-08-26 13:48:09 +02:00
amercader 30f19713fa Add hook to extend the package dict in CKAN harvester 2019-08-26 13:38:30 +02:00
amercader 7bad691415 Add tests and docs for include/exclude groups 2018-10-25 13:04:37 +02:00
amercader 3e5f98e708 Document redis password param 2018-10-25 12:20:47 +02:00
Stefanie Taepke 304bbcec06 send harvest-error-mails to organization-admins 2018-06-14 11:07:49 +02:00
amercader 9085533909 Readme tweaks 2018-06-13 14:19:30 +02:00
Stefanie Taepke c29b52867c update readme for receivers of error-mails 2018-06-13 13:54:13 +02:00
Stefanie Taepke 5f414e2c99 add error-notifications in case of harvest-failure 
requires configuration-values:
ckan.harvest.status_mail.errored = true
email_to = %comma seperated list of emails to receive the email%
error_email_from = %sender of email%
ckan.site_title = %will be included in the email%
 2018-06-11 13:38:38 +02:00
Stefan Oderbolz 2619185bf9
Merge pull request #318 from ckan/readme-test-tip-ckan-parameter 
Readme test tip ckan parameter
 2018-06-07 18:57:09 +02:00
seitenbau-govdata 431d202d4e Add tests and documentation for dataset name suffix config 2018-06-01 21:45:29 +02:00
David Read 47bdd576fb Minor formatting improvements - unquote 2018-04-03 12:24:33 +01:00
David Read a11c2a6697 Minor formatting improvements 2018-04-03 12:20:18 +01:00
David Read c2000e3804 Added a tip, based on running related tests in ckanext-dcat. 2018-04-03 12:13:03 +01:00
Stefan Oderbolz 291d573870 Add a "Releases" section to README 2017-03-30 23:50:39 +02:00
Stefan Oderbolz a0562de442 Update README to link to CONTRIBUTING 
Add link to CKAN guidlines in CONTRIBUTING
 2017-03-30 23:25:25 +02:00
Davor Lozic 863305ece8 Fix JSON typo 2017-01-05 12:31:40 +01:00
rnoerenberg ff1b861f1b Update documentation 
Added note with the limit of 1000 harvest sources
 2016-11-16 16:02:12 +01:00
Raphael Stolt e8570b9e50 Add clearsource history command 2016-11-15 15:04:01 +01:00
David Read ab6a83c49a [#260] Fix the ckan ini filename 2016-08-05 16:12:59 +01:00
David Read d0a8cab479 [#260] Improve install docs 2016-07-29 12:05:12 +01:00
David Read 9d1c20fc44 [#249] Slight README improvement. 2016-06-13 10:28:18 +00:00
David Read f1742fb51a Fix default_groups. It accepted a list of package_name/ids and was trying to add this to the package, but the package needs a dict. Added test. 2016-06-10 09:16:32 +00:00
David Read bfc9b8e0d9 [#249] Test and fix docs for default_tags. Needed to improve error handling when saving ValidationError in a HOE. 2016-06-09 22:11:03 +00:00
Petar Efnushev 1fda3c0255 Change db logger pagination param name 2016-06-06 10:27:05 +02:00
Petar Efnushev 34dedf3869 Add harvest log API usage documentation 2016-06-03 13:55:34 +02:00
Petar Efnushev 9641dc69c2 README updates 2016-05-17 11:31:38 +02:00
Petar Efnushev cc6cb3e389 Changed default config params for the database logger 
Added database logger test case
README updates
 2016-05-16 13:15:12 +02:00
Petar Efnushev a3026f915a README updates 2016-05-12 09:46:33 +02:00
Petar Efnushev ceb8d26aa0 Spacing fixes in README.rst 2016-05-11 17:26:40 +02:00
Petar Efnushev 0be2c868cb README updates 
DBLogHandler updates
Added harvest_log table migration for existing users
Implemented database log scoping
 2016-05-11 13:29:53 +02:00
Petar Efnushev cc86f4062a README updates 2016-05-06 19:07:03 +02:00
Petar Efnushev fe3e92bffe README updates 2016-05-06 18:59:16 +02:00
Petar Efnushev dc4abdcbd8 README updates 2016-05-06 18:54:57 +02:00
Petar Efnushev 30388f04e7 spacing fix 2016-04-11 21:03:40 +02:00
Petar Efnushev 17804b902b spacing fixes 2016-04-11 21:01:11 +02:00
Petar Efnushev 3665889f27 Updated README.rst with configuration options for the logger 2016-04-11 19:42:09 +02:00
David Read 4c1f27161e Fix merge 2016-02-12 18:22:36 +00:00
David Read 7096b7ddf2 Merge branch 'master' of github.com:ckan/ckanext-harvest into 157-version-three-apify 2016-02-12 16:51:26 +00:00
David Read 260cd1f2b7 Merge branch 'master' of github.com:ckan/ckanext-harvest into 214-remove-genshi 2015-12-10 16:02:50 +00:00
David Read b4370f35b7 Note that CKAN 2 is the minimum version now genshi is removed. 2015-12-10 16:02:07 +00:00
David Read f67029b993 Hint about rerunning the import_stage. 2015-12-01 17:59:11 +00:00
David Read b8c7d39e1a Document existing functionality for aborting during the gather stage. 2015-12-01 17:56:24 +00:00
David Read 25301e2152 Update README from interfaces.py. PEP8. Mention HarvestObject - Package relation in interface. 2015-12-01 17:51:35 +00:00
amercader f1ba2bcfb3 Namespace Redis keys to avoid conflicts between instances 
The `ckan.site_id` config option (or `default` if missing) is used to
namespace the Redis keys: routing key and persistance key. Consumers
will only get the relevant keys for their instance.
 2015-11-20 14:17:25 +00:00
amercader ede50aa3fb Merge branch 'immediate-harvest' 2015-11-20 12:28:35 +00:00
David Read 8a7bc9e1d8 Merge remote-tracking branch 'origin/master' into immediate-harvest 
Conflicts:
	README.rst
	ckanext/harvest/commands/harvester.py
	ckanext/harvest/logic/action/create.py
	ckanext/harvest/logic/action/update.py
	ckanext/harvest/logic/auth/update.py
 2015-11-03 00:40:25 +00:00
David Read c7fac36c1c [#107] "unchanged" response tested and related fixes 
* fix "existing_package_dict" which wasn't containing metadata_modified (because of the schema in the context) so you never skipped an object.
* fix IntegrityError due to resource revision_id being harvested. No idea why this hasn't caused errors before now.
* "unchanged" is now checked in base instead of ckanharvester - makes sense. Looking at other harvesters, it's normal to return from the import_stage with the value returned from base._create_or_update_package so I've continued with that.
* "unchanged" response is now documented
* better report_status tests in test_queue2.
 2015-11-03 00:22:53 +00:00
David Read 14f372aec6 Merge branch 'master' of github.com:ckan/ckanext-harvest into 157-version-three-apify 
Conflicts:
	README.rst
 2015-11-02 17:01:22 +00:00
amercader 3f856c3b8c Merge branch 'activate-pyenv' 2015-10-30 14:12:56 +00:00
David Read f1d2d5fdc4 [#111] Run jobs straight away. 2015-10-28 21:58:36 +00:00