Commit Graph

588 Commits

Author SHA1 Message Date
amercader fcbe6aa6de Script for creating harvest source datasets on old versions
The way we check whether datasets need to be created might need to be
improved.
2012-12-05 18:54:28 +00:00
amercader 22ec9cb5af Fix old controller import 2012-12-05 18:53:35 +00:00
amercader 697933f8d0 Add custom harvest source read page (provisional) 2012-12-05 15:47:02 +00:00
amercader 2dba7fbf78 Add custom harvest sources search page 2012-12-05 14:51:20 +00:00
amercader a605564a41 Fix links to harvest sources page 2012-12-05 13:01:56 +00:00
amercader d77bf255b4 Finish up create and edit forms, including breadcrumbs, links, etc 2012-11-30 18:53:13 +00:00
amercader 9d83322591 Fix config validator and add tests 2012-11-30 17:02:06 +00:00
amercader 803b228d1c Update harvest source create and update logic functions
`harvest_source_create` and `harvest_source_update` now call
`package_create` and `package_update` respectively, making sure to
define a 'harvest_source' type. The returned dict uses the db_to_form
schema.
2012-11-30 14:11:24 +00:00
amercader 0e0aed0503 Clean up schemas
Better naming, remove old ones, ignore __extras field
2012-11-30 13:20:37 +00:00
amercader 875a773f1c Check if type property is actually there 2012-11-30 11:10:21 +00:00
amercader 7db09fceb0 Various fixes for the harvest source dataset type forms
Add a db to form schema to show the fields stored in extras. Validate
the source url on the Package object.
2012-11-29 16:57:20 +00:00
amercader ab7a379058 Behind the scenes creation and updating of HarvestSource objects
Taking advantage of the new after_create/after_update extensions points,
the extension checks if the dataset type is harvest source and creates
or updates the corresponding HarvestSource object. When creating a new
one, it will use the same id as the dataset.
2012-11-29 16:48:44 +00:00
amercader 9d36fd6841 First stub of the new dataset type forms
Adds a 'harvest_source' dataset type that mimics the original harvest
source form.
It works against the 3022 branch on CKAN core.
2012-11-29 12:31:48 +00:00
amercader 866fd69730 Do not remove XML declaration and add utf-8 charset to headers 2012-11-20 15:43:39 +00:00
amercader c52ed3b163 Add line field to object error table 2012-11-20 11:29:58 +00:00
amercader 03fd1884f4 Implement retry times for harvest objects 2012-11-15 18:11:35 +00:00
kindly 202c9d9fcc use correct queue for gather stage 2012-11-15 14:21:09 +00:00
kindly c9c1eb4848 use generator to consume 2012-11-15 14:14:55 +00:00
amercader 33d5e09722 Change fetch_callback to proper acknowledge objects 2012-11-15 11:36:06 +00:00
amercader 13357893ad Fix typo 2012-11-13 14:41:38 +00:00
amercader 54ff0526bb Return original document if present when requesting an object 2012-11-13 12:06:36 +00:00
amercader 820443d58f Add cascade option to harvest object extras and errors 2012-11-09 14:52:34 +00:00
kindly 5063626554 make sure state is changed to error on fetch error 2012-11-07 09:53:16 +00:00
kindly 28e5e9137a add perge queues command 2012-11-07 09:51:25 +00:00
kindly 6db65b5826 made manual default not null 2012-11-05 13:17:32 +00:00
amercader fdf01c09f2 Fix wrong check for harvest sources 2012-11-01 14:12:45 +00:00
amercader d598c0707b Ignore frequency field on the frontend for the time being 2012-11-01 14:12:01 +00:00
amercader d7f8c9165c Merge branch 'model_upgrade' into release-v2.0 2012-10-30 18:07:24 +00:00
amercader d502b925a6 Remove old deprecated tests and some whitespace 2012-10-30 18:07:05 +00:00
amercader a136cbf202 Fix typos in migration script 2012-10-30 17:52:10 +00:00
amercader 61b99e8eff Merge branch 'pika' into release-v2.0 2012-10-30 17:31:30 +00:00
amercader 82a498d9fc Rename function to be implementation independent 2012-10-30 17:13:39 +00:00
amercader 94c4f4c9d5 Update requirements file, add pika and drop carrot 2012-10-30 17:12:44 +00:00
amercader 7a6f6cf735 Merge branch 'master' into release-v2.0 2012-10-30 16:01:41 +00:00
kindly 2529a17304 add jobs at certain frequencies 2012-10-29 17:15:02 +00:00
kindly 9fc0ae9937 add next run field 2012-10-26 10:50:35 +01:00
kindly bc079c6644 model upgrade with tests and migration 2012-10-25 19:01:54 +01:00
kindly 1153c1c5c9 add full queue test and new test harvester 2012-10-24 11:58:00 +01:00
kindly da125cdcc2 pika now used as queue library 2012-10-24 00:34:32 +01:00
amercader 8233b2ec23 Strip spaces from url when creating or updating a source 2012-08-17 12:25:06 +01:00
amercader c1f83e0d3e Strip spaces from url when creating or updating a source 2012-08-17 12:24:41 +01:00
tobes a17e8208de Very small text fix 2012-08-16 09:30:04 +01:00
tobes 7e940b497d Text message minor fix 2012-08-16 09:27:36 +01:00
tobes b6a32fd23b Add descriptions for sources 2012-08-16 09:16:34 +01:00
tobes c984727de5 Minor template tidy 2012-08-16 08:56:25 +01:00
tobes 5b7a9c0855 Flash messages to notices plus translatable 2012-08-16 08:49:35 +01:00
amercader 19ea538097 [#2852,#2853] Reword errors 2012-08-15 18:28:08 +01:00
amercader 7609a93422 Minor css tweaks on the forms 2012-08-15 18:26:36 +01:00
tobes c6c4f6d098 Remove about text placeholder 2012-08-15 10:40:52 +01:00
tobes 3b8075b670 Only specify autoform items once 2012-08-14 18:01:29 +01:00