Commit Graph

26 Commits

Author SHA1 Message Date
amercader 2a07a144fc [#84] Fix auth audit exception when creating datasets
This was caused by a combination of the auth audit leaking and the
harvester reusing the context for the package_show and package_create
actions. If the package is not found, package_show does not call
check_access, and the auth audit does not pass. This is stored in the
context (`__auth_audit`) and is raised next time that we call
get_action (when we call package_create with the same context)

It could potentially be fixed on master, but it is probably quite rare.
2014-02-10 18:22:48 +00:00
Rachel Knowler bf11e4d330 Moved clean_tags check into _create_or_update_package method. 2014-02-10 09:29:01 +01:00
Rachel Knowler 2ba9908653 Config option to munge tags changed to be consistent with other config options in this extension, and noted in README. 2014-01-29 10:55:51 +01:00
Rachel Knowler 5e1aef1d08 Removed extra newline. 2014-01-29 10:06:32 +01:00
Rachel Knowler 7d71b0a00b Wrap tag munging code in config option, defaulting to False. 2014-01-29 10:02:16 +01:00
amercader 0f5624822c Use remote name if present when creating datasets on CKAN harvester 2013-10-11 16:50:25 +01:00
Vitor Baptista f028375ad3 [#62] Use current name when updating package, if the user haven't sent a new one
It's hard for someone outside CKAN to make sure they're sending it in the format
we expect. And they'll also have to keep track of our name format, to keep in
sync whenever we change.

To fix this, we simply do what we already do when creating packages: use a
default name. In this case, the current one.
2013-08-18 12:08:30 -03:00
amercader 39ad78d90a [#59] Ignore auth in the CKAN harvester 2013-08-15 14:37:12 +01:00
Konrad Reiche c858b9fe9f Add exception handling for the API version parsing
I have added try-except clauses in order to prevent the process from
crashing if a non-parsable integer is used for the api_version option.

Signed-off-by: Konrad Reiche <konrad.reiche@fokus.fraunhofer.de>
2013-05-27 13:12:05 +02:00
Konrad Reiche 05094090af Change type of the API version to integer
The CKAN logic uses integers when dealing with the API version, e.g.
making checks which API version is in use. Currently, the harvester
uses strings to identify the API version. Instead of dealing with
type conversion the harvester could use integers directly.

This commit fixes okfn/ckanext-harvest#36. When the API version is
parsed from the configuration it is passed through the int() function.
This way the harvesting will still work even if a harvest source was
configured with a string API version which makes this commit backward
compatible.

Signed-off-by: Konrad Reiche <konrad.reiche@fokus.fraunhofer.de>
2013-05-27 12:51:48 +02:00
kindly c754479014 #29 make new idatasets form work with harvest source form 2013-03-25 17:38:07 +00:00
joetsoi 7257258ca4 mark new harvest objects as current
When a new harvest_object for a new package was being created, it
was immediately being marked as false, as all objects were marked
as false, including the new object just created and newly marked
as current=true.

Fix so that old HarvestObjects are only marked as current=False
when updating an existing package.
2013-03-07 20:27:27 +00:00
joetsoi ba486a9482 add indexing of datasets whilst harvesting 2013-02-27 11:34:09 +00:00
amercader 177349fd76 Update HarvesterBase
This is a convenience class that other harvesters can extend. Updates
include a cleanup of old functions and porting of enhancements from the
spatial harvesters.
2013-02-12 16:10:13 +00:00
amercader f210455aef [ckan harvester] Replace title on default extras 2012-03-13 12:38:14 +00:00
amercader e0bef2ef9c [base] Minor fix for harvesters without config 2012-03-12 14:46:28 +00:00
amercader 479750da09 [#1726][base harvester] Set current field when importing 2012-02-02 13:18:43 +00:00
amercader ae51093213 [ckan harvester] Ignore __junk field, was causing imports to fail 2012-01-10 14:46:12 +00:00
Adrià Mercader da469ab08e [base harvester] Custom tag munge function. TODO: check with flexible tags 2011-11-23 11:05:52 +00:00
Adrià Mercader 0ab5c53b47 [ckan harvester] Fix typo 2011-11-18 17:53:01 +00:00
Adrià Mercader c939d90dbb [ckan harvester] Support for defining a custom user to do the harvesting 2011-11-18 14:12:30 +00:00
Adrià Mercader 2018d9e513 [ckan harvester] Support for default tags and groups 2011-11-18 13:20:41 +00:00
Adrià Mercader c04d80e27e Use get_action function instead of directly calling the action functions 2011-10-26 17:26:18 +01:00
Adrià Mercader 7927329536 Make harvesters work with latest ckan release 2011-07-29 11:31:03 +01:00
Adrià Mercader c867660e7d Add docs to base harvester functions 2011-07-18 17:35:03 +01:00
Friedrich Lindenberg 89934b8538 [harvesters] factor out a base harvester for use in generic harvesting apps 2011-06-02 12:07:07 +02:00