Commit Graph

501 Commits

Author SHA1 Message Date
amercader fbde0b8dc1 [#87] Remove remote url_type from resources
Otherwise CKAN thinks they are uploads, datastore resources, etc, which
it can cause problems eg when displaying the URL of the resource. We
are just linking to the remote resource URL.
2014-02-11 17:27:19 +00:00
amercader 5739e541d7 [#80] Support for Python 2.6 when handling xml exceptions 2014-02-10 18:44:46 +00:00
amercader 2a07a144fc [#84] Fix auth audit exception when creating datasets
This was caused by a combination of the auth audit leaking and the
harvester reusing the context for the package_show and package_create
actions. If the package is not found, package_show does not call
check_access, and the auth audit does not pass. This is stored in the
context (`__auth_audit`) and is raised next time that we call
get_action (when we call package_create with the same context)

It could potentially be fixed on master, but it is probably quite rare.
2014-02-10 18:22:48 +00:00
amercader 5b677b6099 [#83] Fix key error when using default_groups 2014-02-10 13:16:58 +00:00
Rachel Knowler bf11e4d330 Moved clean_tags check into _create_or_update_package method. 2014-02-10 09:29:01 +01:00
Rachel Knowler 2ba9908653 Config option to munge tags changed to be consistent with other config options in this extension, and noted in README. 2014-01-29 10:55:51 +01:00
Rachel Knowler 5e1aef1d08 Removed extra newline. 2014-01-29 10:06:32 +01:00
Rachel Knowler 7d71b0a00b Wrap tag munging code in config option, defaulting to False. 2014-01-29 10:02:16 +01:00
amercader 2b803a3f66 [#77] Use auth_allow_anonymous_access decorator
Starting from 2.2 you need to explicitly flag auth functions that
allow anonymous access with the p.toolkit.auth_allow_anonymous_access
decorator. A local version of the decorator is used to ensure we only
use it on CKAN>=2.2
2014-01-20 13:47:37 +00:00
amercader 4cc56f51ab [#76] Use harvest_source_show on reindex command 2014-01-14 17:04:34 +00:00
amercader 95d0ef0f01 [#76] Add extra fields to the source schema
Add 'private' and its core validators, and 'metadata_modified' and
'metadata_created'.

Also ignore '__extras'
2014-01-14 17:01:25 +00:00
amercader 467fb7bb8f Fix resource updating for harvested datasets
Starting from 2.2, resource_update calls package_show before updating
the resource via a package_update call. The dict passed had the harvest
extras (eg harvest_object_id) added which made the update call fails due
to duplicated extra keys. To fix it we now remove any harvest extras
on after_show if there is a 'for_edit' property on the context.
2014-01-13 10:30:52 +00:00
amercader 278a8e1ada Merge branch 'master' of github.com:okfn/ckanext-harvest 2014-01-10 13:49:38 +00:00
amercader 1e94a11255 [#70] Fix Add harvest source button not showing
Due to changes in the templates starting on 2.1 the add source button
was not showing. The whole search template has been simplified,
separating in a separate file the 2.0 only code.

Tested in 2.0, 2.1 and 2.2
2014-01-10 13:48:02 +00:00
Mikko Koho 51e842ee6e Add quotes to harvest_source_id in Solr query when clearing harvest sources 2013-11-22 11:01:26 +02:00
Mikko Koho c338452872 Put harvest_source_id in quotes to prevent Solr errors 2013-11-21 14:14:56 +02:00
amercader 928ea061aa Improve organizations dropdown on source form 2013-10-24 12:33:44 +01:00
Stefan Oderbolz c52085006a [#61] Truly ignore harvest sources
The currently implementation returns False when a harvest source is being harvested. This leads to an error on the harvesting job, which in turn tends to confuse users that have no idea of this special implementation. This fix ensures that harvest sources are still ignored, but silently.
2013-10-23 07:40:55 +02:00
amercader c18d9dc3af [#71] CKAN harvester: Add datasets to source organization
If the harvest source belongs to an organization, new datasets should be added
to it. This is already the case in the spatial harvesters.

The remote orgs logic has been kept, with the difference that if for
some reason the remote org can not be assigned, the local one is used.

If the source does not have an organization, none is added.
2013-10-22 16:24:43 +01:00
amercader 380c14c22c Fix CLI sources list output 2013-10-16 13:03:32 +01:00
amercader 55d2b4e304 Fix purge command 2013-10-16 12:59:23 +01:00
amercader bd62b62764 Merge branch 'metaodi-add-harvesting-of-organizations' 2013-10-15 17:50:04 +01:00
amercader 49999893e7 Merge branch 'add-harvesting-of-organizations' of git://github.com/metaodi/ckanext-harvest into metaodi-add-harvesting-of-organizations 2013-10-15 17:49:28 +01:00
Stefan Oderbolz 8b5d70c6fe Only try to create/match a organization if there is a remote_org 2013-10-11 18:08:32 +02:00
amercader 0f5624822c Use remote name if present when creating datasets on CKAN harvester 2013-10-11 16:50:25 +01:00
amercader 340e9eed63 Merge branch 'add-harvesting-of-organizations' of git://github.com/metaodi/ckanext-harvest into metaodi-add-harvesting-of-organizations 2013-10-11 16:14:18 +01:00
amercader e9dde3f48a Only show the 'Add Harvest Source' button if user is authorized 2013-10-11 11:55:14 +01:00
Stefan Oderbolz dd1acd0c6b Use remote_orgs for organizations 2013-10-07 11:22:19 +02:00
amercader f89f12203c Merge branch 'fix/rename-ampq-to-amqp' of git://github.com/opendatatrentino/ckanext-harvest into opendatatrentino-fix/rename-ampq-to-amqp 2013-10-04 17:24:53 +01:00
amercader c5f4d6889b Merge branch 'harvest-object-create' 2013-10-04 17:22:58 +01:00
Stefan Oderbolz d50eb6fca8 Harvesting of remote organisations similar to remote groups 2013-10-04 16:37:52 +02:00
joetsoi da2fd45e80 [#65] make harvest_job_exists validator return model object
return the model in the validator instead of checking that it exists in
the validator, returning the id and then fetching it again in the action
function
2013-10-03 15:51:37 +01:00
Samuele Santi 611b9aab6d Fixed typo: ampq -> amqp 2013-09-19 11:43:03 +02:00
joetsoi 9b3199b41b [#65] remove unused code 2013-09-17 17:02:38 +01:00
joetsoi 5da153c6b6 [#65] harvest_object_create action
update to use schema and validators. Also accept more parameters to
data_dict.
2013-09-17 16:49:19 +01:00
John Martin 71aedf3fd4 Makes ede45bd work in both CKAN 2.0 and 2.1 2013-09-05 15:36:00 +01:00
John Martin ede45bd1be Fixes #66 by correcting the block name 2013-09-05 15:31:44 +01:00
joetsoi 1b663bbff4 add harvest_object_create action 2013-09-04 14:17:01 +01:00
amercader 52956feab9 Merge branch '62-default-package-name' 2013-08-19 18:23:37 +01:00
amercader f51b8e905a [#58] Check properly for version numbers (patch numbers) 2013-08-19 18:13:01 +01:00
Vitor Baptista f028375ad3 [#62] Use current name when updating package, if the user haven't sent a new one
It's hard for someone outside CKAN to make sure they're sending it in the format
we expect. And they'll also have to keep track of our name format, to keep in
sync whenever we change.

To fix this, we simply do what we already do when creating packages: use a
default name. In this case, the current one.
2013-08-18 12:08:30 -03:00
John Martin 86dcd933ea Merged master 2013-08-15 18:47:16 +01:00
John Martin 712e150b52 [#58] Fix to make merge nice 2013-08-15 18:43:46 +01:00
John Martin 575df637b4 [#58] Fixes to make harvest templates to work with both CKAN 2.0 and 2.1 2013-08-15 16:45:02 +01:00
amercader 05e6362c38 Merge branch 'fix-jinja-status-exception' of git://github.com/metaodi/ckanext-harvest into metaodi-fix-jinja-status-exception 2013-08-15 14:39:20 +01:00
amercader 01ca5c0dfd [#61] Ignore harvest sources on the CKAN harvester 2013-08-15 14:38:33 +01:00
amercader b25fffda93 [#36] Fix bug on API version checking 2013-08-15 14:37:55 +01:00
amercader 39ad78d90a [#59] Ignore auth in the CKAN harvester 2013-08-15 14:37:12 +01:00
Stefan Oderbolz f26baf6c09 Hide both the label and the number of datasets when 'status' is not available 2013-08-15 13:25:16 +02:00
amercader 1c36b33aaf [#59] Ignore auth when using site_user 2013-08-14 12:28:27 +01:00