da125cdcc2
pika now used as queue library
2012-10-24 00:34:32 +01:00
8233b2ec23
Strip spaces from url when creating or updating a source
2012-08-17 12:25:06 +01:00
c1f83e0d3e
Strip spaces from url when creating or updating a source
2012-08-17 12:24:41 +01:00
a17e8208de
Very small text fix
2012-08-16 09:30:04 +01:00
7e940b497d
Text message minor fix
2012-08-16 09:27:36 +01:00
b6a32fd23b
Add descriptions for sources
2012-08-16 09:16:34 +01:00
c984727de5
Minor template tidy
2012-08-16 08:56:25 +01:00
5b7a9c0855
Flash messages to notices plus translatable
2012-08-16 08:49:35 +01:00
19ea538097
[#2852,#2853] Reword errors
2012-08-15 18:28:08 +01:00
7609a93422
Minor css tweaks on the forms
2012-08-15 18:26:36 +01:00
c6c4f6d098
Remove about text placeholder
2012-08-15 10:40:52 +01:00
3b8075b670
Only specify autoform items once
2012-08-14 18:01:29 +01:00
e1c74bdbe6
Fixes for autoform extra_text
2012-08-14 17:56:49 +01:00
8f6bab104e
Dirty form changes pending cleanup
2012-08-14 17:33:32 +01:00
1979517706
Widen url field
2012-08-14 12:02:02 +01:00
a76140650d
Merge branch 'release-v2.0' of github.com:okfn/ckanext-harvest into release-v2.0
2012-08-14 11:40:35 +01:00
4b68e4c31b
Fix details page template and style
2012-08-14 11:23:56 +01:00
eb12152089
Fix index page template and style
2012-08-14 11:04:17 +01:00
4b4e5dba62
fix broken show form
2012-08-14 00:44:00 +02:00
7efca28c22
Template updates
2012-08-10 13:05:54 +01:00
5557da653f
First draft of new source page
2012-08-10 10:06:37 +01:00
3feca92d55
First draft of index page
2012-08-10 10:00:02 +01:00
d8a98fd64a
Move to new plugins model
2012-08-10 09:59:18 +01:00
a8aebac965
Fix the harvest object show call
2012-08-09 13:38:17 +01:00
bb5ba43ebb
Allow showing harvest objects by default (on the default auth profile)
2012-08-09 13:37:28 +01:00
4c562e5f5f
Do not store the object when importing
2012-08-09 11:17:41 +01:00
e4b3cb440c
Do not use repo.are_tables_created
...
When checking whether the core tables have been alredy created it is
best to use package_table.exists(), as are_tables_created reflects the
tables, causing conflicts with other extensions.
This allows ckanext-harvest and ckanext-spatial to be used together on
ckan 1.8 onwards.
2012-08-09 11:06:05 +01:00
4d2fdeac57
Allow defining segments of harvest objects to import
...
Useful when importing large number of objects, as it allows
parallelization
2012-08-02 18:41:59 +01:00
7011efe5dc
Allow not linking to datasets when importing records
...
With the -j flag, harvest objects are not linked to datasets when
importing. This is useful sometimes when importing records for the first
time.
2012-07-30 12:11:55 +01:00
203bcb053b
Status can have links in it now.
2012-07-23 16:15:11 +01:00
a61ea06faf
Merge branch 'master' of github.com:okfn/ckanext-harvest
2012-07-19 15:27:04 +01:00
1a4e43a2a9
Status message added - change config to set the text.
2012-07-19 15:17:50 +01:00
4d00e665f1
[cli] Speed up run command
2012-06-29 11:32:18 +01:00
5df2b64dda
Merge branch 'master' of github.com:okfn/ckanext-harvest
2012-06-15 18:38:33 +01:00
c0a9965b52
Reword warning.
2012-06-15 18:38:22 +01:00
528e98120c
[ #2533 ] Fix some imports broken by ckan cleanups
2012-06-15 12:08:35 +02:00
ccf0cd3da2
Add copious logging to record what happens in harvesting.
2012-06-08 17:09:22 +01:00
a5fac2ac86
Added logic for getting a harvest source when viewing a dataset.
2012-06-01 17:03:40 +01:00
5151f4ee23
In publisher auth mode, any member of the group can make the changes. This brings things in line with the general idea that Admins have the power to do this plus authorize other editors/admins.
2012-05-29 15:21:34 +01:00
017222afd2
Withdrawn language introduced for uklp.
2012-05-28 18:41:03 +01:00
103eea9d50
Improve layout of sources to span page.
2012-05-28 18:13:34 +01:00
b1a4cc7721
[ui][xs] Nicer button
2012-05-10 11:53:19 +01:00
3941b691f4
[ui] Fix style for CKAN 1.7
2012-05-09 16:03:03 +01:00
f0a09e8299
[logic] Fix small bug in dictization
2012-05-09 15:58:23 +01:00
718202d886
Logging is now assured.
2012-04-10 20:53:29 +01:00
00e911a70c
Fix name of the queue logger. Moved imports of ckanext.harvest until after _load_config so that the loggers do not start disabled.
2012-04-10 20:10:17 +01:00
e797f50a05
[cli] Fix create job command
2012-03-19 17:28:53 +00:00
38a9a03355
[logic] Fix variables naming
2012-03-19 17:01:20 +00:00
1145e6ea72
[master][auth/publisher] Check for 'ignore_auth' in harvest_object_show
...
Use case: In ckanext-dgu we want to index the harvest_object.content field. As indexing is done synchronously we need
to provide a way for that harvest_object to be accessed when the current http request is made by a non-sysadmin user.
2012-03-15 18:14:57 +00:00
7f10418f44
[master][auth] get_obj_object() function was missing
2012-03-15 18:09:44 +00:00
871eae94b6
[ckan harvester] Fix bug on force all check
2012-03-15 11:31:12 +00:00
f210455aef
[ckan harvester] Replace title on default extras
2012-03-13 12:38:14 +00:00
e0bef2ef9c
[base] Minor fix for harvesters without config
2012-03-12 14:46:28 +00:00
4fe38ec49d
[tests,auth] Add tests for the auth profiles
...
Note that only the tests related to the currently loaded auth
profile will be run.
2012-03-08 16:14:44 +00:00
4a7007460b
[logic] Fix broken imports
2012-03-07 17:08:17 +00:00
763f07fcad
[logic,cli] Add session to the context in cli commands
2012-03-07 15:20:49 +00:00
bf6df2dcd6
Fix merge
2012-03-07 15:04:50 +00:00
9fcaefe8ff
[ui] Fix source datasets paging
2012-03-07 15:03:33 +00:00
6cccbb61c9
Bug fix, new job count property had not been updated
2012-03-07 12:10:32 +00:00
124f3191c8
[ui] Add class to config fields so they can be hidden via CSS
2012-03-07 11:56:18 +00:00
d9cfc52643
[ui,auth] Aggregate sources by publisher on the sources list
2012-03-07 11:49:12 +00:00
97b390f3c1
[auth,logic,ui] Handle publishers on the UI
...
Add fields for publishers in the form when using the publihser auth
profile. Some changes related to the source schema.
2012-03-06 16:01:43 +00:00
aea785701f
[logic,auth] Check that users actually exist
2012-03-06 10:37:31 +00:00
f0e2521d9b
[logic,auth] Modify checks to ensure users are admins of their publishers
2012-03-06 10:16:27 +00:00
d98206858d
[plugin,auth] Check on startup if ckan is also using the publisher profile
2012-03-05 17:10:02 +00:00
2a2397c0ed
[logic,auth] Implement publisher auth profile
...
The publisher profile allows general users to handle harvest sources
based on membership to a certain group (publisher), as opposed to the
default auth profile where only sysadmins can perform any harvesting
task.
To enable it, put this directive in your ini file:
ckan.harvest.auth.profile = publisher
TODO:
* Save publisher id / user id when creating sources
* Show publisher in form and index page
2012-03-02 16:49:39 +00:00
3b68298bba
[logic,auth] Use the site user for CLI commands auth checks
2012-03-01 12:46:42 +00:00
a35eb75440
[logic,auth] Add auth logic layer
...
The first version of the auth layer is based on the current policy, i.e.
you need to be sysadmin to perform any action.
TODO: the CLI is still not working.
2012-03-01 12:02:16 +00:00
c798013752
[logic] Refactor the rest of the logic functions (create,update,delete)
2012-02-29 15:20:35 +00:00
651474e9f1
[logic] Refactor logic layer to follow CKAN core conventions
...
To make maintenance easier and better support the upcoming auth checks,
the logic layer has been refactored to mimic the structure of the one on
CKAN core: separate actions and dictize functions and logic functions
receive a context.
Only get functions are included in this commit.
2012-02-29 10:59:02 +00:00
50537a6738
Merge branch 'master' into enh-1726-harvesting-model-update
2012-02-15 12:01:15 +00:00
e03c2545ca
[ui,logic] Expose source title in the source form
2012-02-15 11:49:59 +00:00
3489a004ad
[ui] Minor tweak to support older themes
2012-02-14 17:23:17 +00:00
2990353533
[ui,logic] Expose source state (active/inactive) in the source form
2012-02-14 14:24:32 +00:00
4d7b8143b9
[lib] Renable unique constraint in url for inactive sources
2012-02-14 11:28:11 +00:00
9ed152cbea
[ckan harvester] Add support for forcing gathering of all remote packages
2012-02-03 17:54:34 +00:00
a5cf445fa6
[ #1727 ][lib] Use 'current' field in queries returning harvest objects
2012-02-02 13:20:03 +00:00
479750da09
[ #1726 ][base harvester] Set current field when importing
2012-02-02 13:18:43 +00:00
4c81c7c3a7
[ #1726 ][model] Harvest source reference compatibility
...
The 'source' property of harvest objects now comes from the actual
foreign key. For compatibility with old harvesters, an before insert
event listener has beeen added to check if the source id has been set,
and set it automatically from the job if not.
Note that this requires SQLAlchemy 0.7 (ie CKAN 1.5.1)
2012-02-01 12:52:52 +00:00
004210935a
[model] Avoid unicode warning
2012-02-01 11:10:44 +00:00
b64d97118c
[ #1726 ][model] Add scripts for populating source_id and current fields
2012-02-01 11:08:41 +00:00
d1783f5415
[model] Changes in harvest model
...
Added three changes to the harvest model:
* 'title' column in harvest_source table
* 'current' column in harvest_job table
* foreign key from harvest_object to harvest_source
Tables are checked on startup to see if they need to be updated.
TODO: populate current and harvest_source_id fields
2012-01-30 18:38:35 +00:00
f086e908bc
[model] Clearer table initialization
2012-01-30 17:09:28 +00:00
a997e45470
[lib] Ignore deleted packages in source stats
2012-01-25 17:47:35 +00:00
3a489bbb82
[ui] Cleanup sources list and details page
2012-01-24 16:55:47 +00:00
0f8c607187
[tests]: Another test moved in wholesale from dgu repo.
2012-01-11 10:35:37 +00:00
81ed69c4da
[tests]: Moved to this repo test code from dgu repo that might be useful or might not. Completely broken, but maybe be worth something.
2012-01-11 10:29:05 +00:00
a53b79c181
[ui] Show edit and refresh links in source page
2012-01-10 17:24:05 +00:00
2ad29df5c5
[lib] Fix bug: couldn't delete source conf
2012-01-10 17:15:56 +00:00
eb646b3385
[ckan harvester] Add support for defining default extras
2012-01-10 17:07:19 +00:00
ae51093213
[ckan harvester] Ignore __junk field, was causing imports to fail
2012-01-10 14:46:12 +00:00
da469ab08e
[base harvester] Custom tag munge function. TODO: check with flexible tags
2011-11-23 11:05:52 +00:00
cfaba6e1e8
[ckan harvester] Add support for sending an API key
2011-11-21 17:29:10 +00:00
0ab5c53b47
[ckan harvester] Fix typo
2011-11-18 17:53:01 +00:00
f02ee45aae
[ui] Show config options in harvest source details page
2011-11-18 14:35:46 +00:00
994590531e
[ckan harvester] Support for creating read-only packages
2011-11-18 14:30:10 +00:00
c939d90dbb
[ckan harvester] Support for defining a custom user to do the harvesting
2011-11-18 14:12:30 +00:00
2018d9e513
[ckan harvester] Support for default tags and groups
2011-11-18 13:20:41 +00:00
8ec05bc3e3
Modify import command to avoid problems with the Session
2011-11-15 11:26:24 +00:00
c04d80e27e
Use get_action function instead of directly calling the action functions
2011-10-26 17:26:18 +01:00
kindly
amercader
tobes
David Read
Sean Hammond
David Read
Ian Murray