CatalogueRevamping
/
harvester-d4science
14
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
935 Commits 3 Branches 0 Tags 2.5 MiB
Commit Graph

935 Commits

Author SHA1 Message Date
Stefan Oderbolz 8b081e2868 Add new parameter return_last_job_status to harvest_source_list 
In order to get a quick overview over successful/failed harvesters, a
call to harvest_source_list with return_last_job_status=true can be used
to get this information.
By default return_last_job_status is False, and hence the extra
resources to grab this information is not wasted for every call, but
only if the client requests is explicitly.
The original 'status' field stays as-is, this PR introduces a new field
called 'last_job_status' to return this information.

The returned information is gather by a call to
harvest_source_status_show.
 2016-08-25 17:56:51 +02:00
Adrià Mercader b0f2c84806 Merge pull request #261 from ckan/260-docs 
[#260] Improve install docs
 2016-08-08 12:21:20 +01:00
David Read ab6a83c49a [#260] Fix the ckan ini filename 2016-08-05 16:12:59 +01:00
David Read d0a8cab479 [#260] Improve install docs 2016-07-29 12:05:12 +01:00
David Read 2208da6669 Merge pull request #258 from torfsen/257-allow-sharing-of-redis-database 
[#257] Purge only our own Redis data.
 2016-07-20 15:58:17 +01:00
Florian Brucker 2602de9094 [#257] Purge only our own Redis data. 
Previously purging the queue on the Redis backend would clear the whole
database, making it hard to share the same database with other parts of
CKAN. With this commit, only the keys that belong to ckanext-harvest and
the current CKAN instance are purged.
 2016-07-20 16:24:13 +02:00
Adrià Mercader f8cfc63885 Merge pull request #255 from ckan/253-default-groups 
[#253] Fix default_groups
 2016-07-01 20:45:46 +02:00
David Read 78933fb775 [#253] Fix default_groups by saving the dicts to the config object, since saving it to the harvester object doesnt work in the real world. This is a lot more efficient than doing group_show for every dataset imported. 2016-06-27 12:01:35 +01:00
Adrià Mercader b737a419dc Merge pull request #252 from jweyrich/master 
Don't let the user specify mutually exclusive configuration options
 2016-06-15 10:06:02 +01:00
Jardel Weyrich e8f539a45e Don't let the user specify mutually exclusive configuration options: 
- organizations_filter_include
- organizations_filter_exclude
 2016-06-14 11:35:38 -03:00
Adrià Mercader f2d8a5f8cc Merge pull request #251 from ckan/249-default-tag 
Fix default_groups, plus fix docs for default_tags
 2016-06-13 13:59:54 +01:00
David Read 9d1c20fc44 [#249] Slight README improvement. 2016-06-13 10:28:18 +00:00
David Read 836605097f Fix test_queue2 tests 2016-06-13 10:12:30 +00:00
David Read 18a506a112 [#249] Add test for default_extras. 2016-06-10 09:51:17 +00:00
David Read f1742fb51a Fix default_groups. It accepted a list of package_name/ids and was trying to add this to the package, but the package needs a dict. Added test. 2016-06-10 09:16:32 +00:00
David Read bfc9b8e0d9 [#249] Test and fix docs for default_tags. Needed to improve error handling when saving ValidationError in a HOE. 2016-06-09 22:11:03 +00:00
Adrià Mercader aeab60ece6 Merge pull request #247 from keitaroinc/change-db-logger-pagination-param-name 
Change db logger pagination param name
 2016-06-06 09:45:57 +01:00
Petar Efnushev 1ba5c4389b Merge branch 'master' into change-db-logger-pagination-param-name 2016-06-06 10:29:55 +02:00
Petar Efnushev 1fda3c0255 Change db logger pagination param name 2016-06-06 10:27:05 +02:00
Petar Efnushev 92120ca47e Change db logger pagination limit param name 2016-06-06 10:23:57 +02:00
Petar Efnushev 34dedf3869 Add harvest log API usage documentation 2016-06-03 13:55:34 +02:00
amercader 13d5395bf0 Bump version 2016-05-23 16:00:23 +01:00
David Read a5c3d2a92c Merge pull request #243 from ckan/240-dont-reuse-contexts 
Don't reuse contexts on ckan harvester
 2016-05-23 14:59:08 +01:00
amercader 5e1512f717 Don't reuse contexts on ckan harvester 
Reusing the same context on all calls can lead to hard to debug failures
like

Action function organization_show did not call its auth function

In this case that was caused because the first organization/group_show
raised a NotFound so the auth audit was still in the context. When
organization/group_show was called again at the end of
organization/group_create the auth audit exception was raised.

This commit makes sure that each call has its own context.
 2016-05-23 12:20:08 +01:00
amercader 3ddce922e0 Merge branch 'keitaroinc-240-fix-groups-import' 2016-05-23 10:21:02 +01:00
amercader 16a6e9fbf6 Add tests for group creation during harvesting 2016-05-23 10:20:52 +01:00
amercader 314be8bc33 Merge branch '240-fix-groups-import' of https://github.com/keitaroinc/ckanext-harvest into keitaroinc-240-fix-groups-import 2016-05-23 09:59:32 +01:00
Petar Efnushev c16ecea7f0 reverted change in default groups validation 2016-05-20 20:15:54 +02:00
Petar Efnushev c154365371 Fixed creation/import of groups and organizations when harvesting from remote ckan instance 2016-05-20 16:38:48 +02:00
Adrià Mercader 1ec2af0590 Merge pull request #230 from keitaroinc/logging-module 
Logging module
 2016-05-17 14:12:03 +01:00
Petar Efnushev 9641dc69c2 README updates 2016-05-17 11:31:38 +02:00
Petar Efnushev cc6cb3e389 Changed default config params for the database logger 
Added database logger test case
README updates
 2016-05-16 13:15:12 +02:00
Petar Efnushev a3026f915a README updates 2016-05-12 09:46:33 +02:00
Petar Efnushev ceb8d26aa0 Spacing fixes in README.rst 2016-05-11 17:26:40 +02:00
Petar Efnushev 0be2c868cb README updates 
DBLogHandler updates
Added harvest_log table migration for existing users
Implemented database log scoping
 2016-05-11 13:29:53 +02:00
amercader 45513c1a38 Merge branch 'pysolr-compatibility' 2016-05-11 10:10:33 +01:00
David Read 623fca5f80 New syntax for pysolr connection 2016-05-10 11:14:35 +00:00
David Read 67bd5046df Merge pull request #237 from Zharktas/fix_syntax_error_in_template 
Fix syntax error in template
 2016-05-10 11:52:30 +01:00
Jari Voutilainen 41c329c6d4 Fix syntax error in template 2016-05-09 14:36:21 +03:00
Petar Efnushev cc86f4062a README updates 2016-05-06 19:07:03 +02:00
Petar Efnushev fe3e92bffe README updates 2016-05-06 18:59:16 +02:00
Petar Efnushev dc4abdcbd8 README updates 2016-05-06 18:54:57 +02:00
Petar Efnushev 009cc57e09 Added clean-up mechanism for the harvest log 2016-05-06 18:44:02 +02:00
Adrià Mercader 74cec5be62 Merge pull request #235 from ckan/other-deprecated-nav_named_link 
Fix deprecated nav_named_link
 2016-04-27 17:54:23 +01:00
David Read e72d7b0267 Merge branch 'master' of github.com:ckan/ckanext-harvest into other-deprecated-nav_named_link 2016-04-27 15:53:48 +00:00
David Read 6539de67e9 Merge pull request #234 from ckan/travis-solr-fix 
Single core solr for travis tests
 2016-04-27 16:53:08 +01:00
David Read cbd65a63ef Single core solr for travis tests. 2016-04-27 15:46:33 +00:00
David Read d372f112f0 Convert deprecated helper. 2016-04-27 15:41:17 +00:00
David Read e1603c2fa4 Merge pull request #233 from Zharktas/create_index_to_harvest_object 
Create index to harvest_object
 2016-04-27 10:11:57 +01:00
Jari Voutilainen 633a32075e create index to harvest_object 2016-04-27 09:27:12 +03:00