Commit Graph

42 Commits

Author SHA1 Message Date
amercader eaa8988440 [#4] Changes in schema to accommodate organizations
Basically handle the 'owner_org' field in form_to_db and db_to_form.
Added 'owner_org', 'frequency' (has default) and 'config' to surplus
keys in check_data_dict.
Also remove schema tweaks to let package_show call the appropiate schema
function.
2013-02-11 16:34:52 +00:00
amercader 8576ad6784 [#5] Add job listing page 2013-02-04 18:20:58 +00:00
amercader 22389fc52a [#5] Update report templates
The job details page has been updated to show the full error report, and
the whole report page has been dropped. All job details are loaded via a
snippet, which is also loaded on the harvest source page.

The frontend is still completely provisional.
2013-02-01 18:32:41 +00:00
amercader d4b6dcb7f6 [#5] Add helper function for generating a link to a harvest object 2013-01-24 18:21:05 +00:00
amercader 30d58b2b7b [#5] Preliminary job report logic function and page (WIP) 2013-01-23 18:04:19 +00:00
amercader 234f9f4cc0 [#5] Add job summary page
Shows dataset and error counts, job details and a summary of the more
frequent errors.
2013-01-23 17:33:44 +00:00
amercader bfce5185f0 [#4] Add db_to_form_schema_options to harvest plugin to avoid validation on show 2013-01-16 17:45:33 +00:00
amercader 2f4cd3a4b0 [#4] Fix logic functions importer 2013-01-15 19:29:17 +00:00
amercader 288e1429a6 [#4] Remove the loading of different authorization profiles
The different profiles will be now configured via the harvest source
datasets on CKAN core, so it is no longer needed.

Also simplify IActions and IAuthFunction hook calls.
2013-01-09 17:32:05 +00:00
amercader 6b23082010 Move logic from setup_template_variables to helper functions 2013-01-09 11:07:44 +00:00
amercader e0f3d47cb9 Add extra information to the harvest source page
The status object gives extra information about the source and there is
a helper function to build the dataset list for this particular source.
TODO: Pager still needs fixing.
2012-12-12 11:54:50 +00:00
amercader b567e562f4 Add after_show extension point
We hook into the package_show extension point in order to:

1. For harvest_source type datasets, add extra information about the
source, jobs, etc (calling harvest_source_show_status)
2. For normal datasets, check if they were harvested, and if so, add a
reference to the harvest object and harvest source.
2012-12-12 11:49:55 +00:00
amercader 2557636994 Update endpoints to receive the context object 2012-12-12 11:47:57 +00:00
amercader 8e1621731b Move harvest source status function as a logic function
The status dict is added automatically to harvest source packages.
Note that the actual queries still need to be updated as they proabably
won't scale.
2012-12-12 11:45:13 +00:00
amercader fcbe6aa6de Script for creating harvest source datasets on old versions
The way we check whether datasets need to be created might need to be
improved.
2012-12-05 18:54:28 +00:00
amercader 697933f8d0 Add custom harvest source read page (provisional) 2012-12-05 15:47:02 +00:00
amercader 2dba7fbf78 Add custom harvest sources search page 2012-12-05 14:51:20 +00:00
amercader d77bf255b4 Finish up create and edit forms, including breadcrumbs, links, etc 2012-11-30 18:53:13 +00:00
amercader 0e0aed0503 Clean up schemas
Better naming, remove old ones, ignore __extras field
2012-11-30 13:20:37 +00:00
amercader 875a773f1c Check if type property is actually there 2012-11-30 11:10:21 +00:00
amercader 7db09fceb0 Various fixes for the harvest source dataset type forms
Add a db to form schema to show the fields stored in extras. Validate
the source url on the Package object.
2012-11-29 16:57:20 +00:00
amercader ab7a379058 Behind the scenes creation and updating of HarvestSource objects
Taking advantage of the new after_create/after_update extensions points,
the extension checks if the dataset type is harvest source and creates
or updates the corresponding HarvestSource object. When creating a new
one, it will use the same id as the dataset.
2012-11-29 16:48:44 +00:00
amercader 9d36fd6841 First stub of the new dataset type forms
Adds a 'harvest_source' dataset type that mimics the original harvest
source form.
It works against the 3022 branch on CKAN core.
2012-11-29 12:31:48 +00:00
tobes d8a98fd64a Move to new plugins model 2012-08-10 09:59:18 +01:00
David Read ccf0cd3da2 Add copious logging to record what happens in harvesting. 2012-06-08 17:09:22 +01:00
David Read a5fac2ac86 Added logic for getting a harvest source when viewing a dataset. 2012-06-01 17:03:40 +01:00
David Read 718202d886 Logging is now assured. 2012-04-10 20:53:29 +01:00
amercader d98206858d [plugin,auth] Check on startup if ckan is also using the publisher profile 2012-03-05 17:10:02 +00:00
amercader 2a2397c0ed [logic,auth] Implement publisher auth profile
The publisher profile allows general users to handle harvest sources
based on membership to a certain group (publisher), as opposed to the
default auth profile where only sysadmins can perform any harvesting
task.
To enable it, put this directive in your ini file:

    ckan.harvest.auth.profile = publisher

TODO:
 * Save publisher id / user id when creating sources
 * Show publisher in form and index page
2012-03-02 16:49:39 +00:00
amercader a35eb75440 [logic,auth] Add auth logic layer
The first version of the auth layer is based on the current policy, i.e.
you need to be sysadmin to perform any action.

TODO: the CLI is still not working.
2012-03-01 12:02:16 +00:00
amercader c798013752 [logic] Refactor the rest of the logic functions (create,update,delete) 2012-02-29 15:20:35 +00:00
amercader 651474e9f1 [logic] Refactor logic layer to follow CKAN core conventions
To make maintenance easier and better support the upcoming auth checks,
the logic layer has been refactored to mimic the structure of the one on
CKAN core: separate actions and dictize functions and logic functions
receive a context.
Only get functions are included in this commit.
2012-02-29 10:59:02 +00:00
Adrià Mercader 63afd199a9 Add link to source documents from object errors 2011-09-08 10:27:36 +01:00
Friedrich Lindenberg 13f2fb3b96 use hasattr for config validation 2011-06-09 11:35:58 +02:00
Adrià Mercader ca6af0249a Reverting previous changeset, as it conflicts with dgu_form_api 2011-06-07 11:58:35 +01:00
Friedrich Lindenberg f9c0ee37aa spacing in template paths 2011-06-06 10:16:34 +02:00
David Raznick 79fd966573 add database setup at configure time 2011-05-31 18:06:26 +01:00
Adrià Mercader bbe459527f [forms] Major refactoring of the harvest forms. Forms no longer use the DGU form
API, and are handled similarly to the new ones on CKAN core (logic, schema,
validators...). The UI is also more consistent with the CKAN one.
2011-05-13 14:17:58 +01:00
james bc8caebc30 [packaging] Moved templates and publics into source 2011-04-18 08:49:25 +00:00
Adrià Mercader a234d6ff6a [refactoring] Remove unneeded interface 2011-04-12 10:11:02 +01:00
Adrià Mercader 842bea8433 [refactoring] Remove all geo related functions as they now have it own extension ckanext-spatial 2011-04-11 19:19:09 +01:00
james gardner e855d2ad9a Moving the extension points code to package as a .deb file 2011-03-28 15:52:43 +01:00