Commit Graph

664 Commits

Author SHA1 Message Date
Stefan Oderbolz dd1acd0c6b Use remote_orgs for organizations 2013-10-07 11:22:19 +02:00
amercader 10b4dec490 Merge branch 'opendatatrentino-fix/rename-ampq-to-amqp' 2013-10-04 17:25:36 +01:00
amercader f89f12203c Merge branch 'fix/rename-ampq-to-amqp' of git://github.com/opendatatrentino/ckanext-harvest into opendatatrentino-fix/rename-ampq-to-amqp 2013-10-04 17:24:53 +01:00
amercader c5f4d6889b Merge branch 'harvest-object-create' 2013-10-04 17:22:58 +01:00
Stefan Oderbolz d50eb6fca8 Harvesting of remote organisations similar to remote groups 2013-10-04 16:37:52 +02:00
joetsoi da2fd45e80 [#65] make harvest_job_exists validator return model object
return the model in the validator instead of checking that it exists in
the validator, returning the id and then fetching it again in the action
function
2013-10-03 15:51:37 +01:00
Samuele Santi 611b9aab6d Fixed typo: ampq -> amqp 2013-09-19 11:43:03 +02:00
joetsoi 9b3199b41b [#65] remove unused code 2013-09-17 17:02:38 +01:00
joetsoi 5da153c6b6 [#65] harvest_object_create action
update to use schema and validators. Also accept more parameters to
data_dict.
2013-09-17 16:49:19 +01:00
John Martin 71aedf3fd4 Makes ede45bd work in both CKAN 2.0 and 2.1 2013-09-05 15:36:00 +01:00
John Martin ede45bd1be Fixes #66 by correcting the block name 2013-09-05 15:31:44 +01:00
joetsoi 1b663bbff4 add harvest_object_create action 2013-09-04 14:17:01 +01:00
amercader 52956feab9 Merge branch '62-default-package-name' 2013-08-19 18:23:37 +01:00
amercader 4c110ea311 Merge branch '58-template-fixes-for-ckan-2.1' 2013-08-19 18:13:16 +01:00
amercader f51b8e905a [#58] Check properly for version numbers (patch numbers) 2013-08-19 18:13:01 +01:00
Vitor Baptista f028375ad3 [#62] Use current name when updating package, if the user haven't sent a new one
It's hard for someone outside CKAN to make sure they're sending it in the format
we expect. And they'll also have to keep track of our name format, to keep in
sync whenever we change.

To fix this, we simply do what we already do when creating packages: use a
default name. In this case, the current one.
2013-08-18 12:08:30 -03:00
John Martin 86dcd933ea Merged master 2013-08-15 18:47:16 +01:00
John Martin 712e150b52 [#58] Fix to make merge nice 2013-08-15 18:43:46 +01:00
John Martin 575df637b4 [#58] Fixes to make harvest templates to work with both CKAN 2.0 and 2.1 2013-08-15 16:45:02 +01:00
amercader d604ce476e Merge branch 'metaodi-fix-jinja-status-exception' 2013-08-15 14:43:16 +01:00
amercader 05e6362c38 Merge branch 'fix-jinja-status-exception' of git://github.com/metaodi/ckanext-harvest into metaodi-fix-jinja-status-exception 2013-08-15 14:39:20 +01:00
amercader 01ca5c0dfd [#61] Ignore harvest sources on the CKAN harvester 2013-08-15 14:38:33 +01:00
amercader b25fffda93 [#36] Fix bug on API version checking 2013-08-15 14:37:55 +01:00
amercader 39ad78d90a [#59] Ignore auth in the CKAN harvester 2013-08-15 14:37:12 +01:00
Stefan Oderbolz f26baf6c09 Hide both the label and the number of datasets when 'status' is not available 2013-08-15 13:25:16 +02:00
amercader 1c36b33aaf [#59] Ignore auth when using site_user 2013-08-14 12:28:27 +01:00
amercader 64c2d14209 Merge branch 'metaodi-fix-add-harvest-source-cli' 2013-08-14 11:56:57 +01:00
amercader ffea49ca62 [#56] Update parameters on source create command
Add missing title and owner_org fields, remove deprecated user_id and
publisher_id
2013-08-14 11:54:51 +01:00
amercader 3494727d3f [#56] Increase max params number 2013-08-14 11:43:32 +01:00
amercader 8e33262026 [#56] Fix syntax error and wrong type 2013-08-14 11:31:23 +01:00
Stefan Oderbolz 4dfd091aec Make the /harvest page more robust if source.status is not set
This prevents exceptions from appearing in the log from Jinja:
  [error] [client 1.2.3.4] Error - <class 'jinja2.exceptions.UndefinedError'>: 'dict object' has no attribute 'status'
2013-08-14 11:52:11 +02:00
Stefan Oderbolz ad0bbea3a0 Added new 'name' parameter to README 2013-08-06 00:03:36 +02:00
Stefan Oderbolz 7ae9d6e208 Made print method more robust against KeyErrors
This is especially needed if you create a new harvest source which does not have all the optional arguments. Before this lead to a KeyError after the creation of the source. Now this simply output 'None'.
2013-08-05 23:50:30 +02:00
Stefan Oderbolz 1249564be5 Add additional name argument when creating new harvest source 2013-08-05 23:46:21 +02:00
Stefan Oderbolz ade5f83e38 Change key of data_dict from 'type' to 'source_type' 2013-08-05 23:07:25 +02:00
amercader cb745c3c3e Avoid importing unnecessary functions from the harvest logic 2013-08-05 18:39:44 +01:00
amercader 18f8a92084 Merge branch 'source-is-a-dict' 2013-07-29 11:22:48 +01:00
Vitor Baptista 70e53a7833 Fix bug where source was being treated as an object, when it's a dict 2013-07-29 07:06:58 -03:00
amercader cc3f3d3426 [#50] Fix objects deletion on gather exceptions 2013-07-05 13:29:11 +01:00
amercader dcb6b964ea Merge branch '50-dates-in-utc' 2013-07-04 15:03:59 +01:00
amercader e2696b98bb [#50] Save all dates as UTC in the database
At some point we may want to transform these to local time at the
dictization level. We will need a library like dateutil to handle it
properly though.
2013-07-04 14:59:27 +01:00
kindly c2283e3fdb only migrate harvest sources which are active 2013-06-28 02:32:45 +01:00
kindly a42991b8c9 fix so that non sysadmins can edit harvest sources of organizations they
are admins or editors of.
2013-06-27 12:16:11 +01:00
kindly 6540726c47 use correct limit for paging harvest listing 2013-06-26 11:14:38 +01:00
amercader 584c340583 Merge branch '42-remove-non-string-extras' 2013-06-03 10:33:59 +01:00
Sean Hammond 01df3a1db4 [#42] Dump non-string extras with json
Convert any non-string extra values to strings using json.dumps(),
instead of just deleting them.
2013-05-31 20:35:06 +02:00
amercader 373046321f Merge branch 'fraunhoferfokus-api-version-fix' 2013-05-31 17:24:48 +01:00
amercader 3a31db59b6 [#36] Move validation code to validate_config
This ensures it is checked whenever the source is edited or created.
2013-05-31 17:23:40 +01:00
amercader a6a0196a4e Merge branch 'api-version-fix' of git://github.com/fraunhoferfokus/ckanext-harvest into fraunhoferfokus-api-version-fix 2013-05-31 17:15:43 +01:00
Sean Hammond 85a013f2c9 [#42] Remove non-string extras from packages
Remove extras whose values are not strings (e.g. dicts, lists..) from
packages before attempting to create or update the packages on the
target site.

In CKAN 1 it was possible for the values of extras to be other types,
but in CKAN 2 they must be strings, so when harvesting from a CKAN 1 site
into a CKAN 2 site SQLAlchemy would crash when trying to create packages
with non-string extras.

The fix in this commit is to simply remove any non-string extras from
the harvested package. (Alternatively, we could try to convert them to a
string using JSON.)

Fixes #42.
2013-05-31 15:43:42 +02:00