d1783f5415
[model] Changes in harvest model
...
Added three changes to the harvest model:
* 'title' column in harvest_source table
* 'current' column in harvest_job table
* foreign key from harvest_object to harvest_source
Tables are checked on startup to see if they need to be updated.
TODO: populate current and harvest_source_id fields
2012-01-30 18:38:35 +00:00
f086e908bc
[model] Clearer table initialization
2012-01-30 17:09:28 +00:00
a997e45470
[lib] Ignore deleted packages in source stats
2012-01-25 17:47:35 +00:00
004ca4a4ee
[model][xs]: Avoid unicode warning.
2012-01-25 11:35:59 +00:00
3a489bbb82
[ui] Cleanup sources list and details page
2012-01-24 16:55:47 +00:00
0f8c607187
[tests]: Another test moved in wholesale from dgu repo.
2012-01-11 10:35:37 +00:00
81ed69c4da
[tests]: Moved to this repo test code from dgu repo that might be useful or might not. Completely broken, but maybe be worth something.
2012-01-11 10:29:05 +00:00
38c7742b08
[doc] Fix typos
2012-01-10 17:55:45 +00:00
eb591521ff
Merge branch 'feature-new-ckan-harvester-features'
2012-01-10 17:27:33 +00:00
a53b79c181
[ui] Show edit and refresh links in source page
2012-01-10 17:24:05 +00:00
2ad29df5c5
[lib] Fix bug: couldn't delete source conf
2012-01-10 17:15:56 +00:00
eb646b3385
[ckan harvester] Add support for defining default extras
2012-01-10 17:07:19 +00:00
ae51093213
[ckan harvester] Ignore __junk field, was causing imports to fail
2012-01-10 14:46:12 +00:00
a971f7be79
[doc]: Convert to github.
2011-12-12 11:10:11 +00:00
ac9a9100f8
[doc] Document api_key config option
2011-11-23 11:09:16 +00:00
da469ab08e
[base harvester] Custom tag munge function. TODO: check with flexible tags
2011-11-23 11:05:52 +00:00
cfaba6e1e8
[ckan harvester] Add support for sending an API key
2011-11-21 17:29:10 +00:00
0ab5c53b47
[ckan harvester] Fix typo
2011-11-18 17:53:01 +00:00
f02ee45aae
[ui] Show config options in harvest source details page
2011-11-18 14:35:46 +00:00
994590531e
[ckan harvester] Support for creating read-only packages
2011-11-18 14:30:10 +00:00
c939d90dbb
[ckan harvester] Support for defining a custom user to do the harvesting
2011-11-18 14:12:30 +00:00
2018d9e513
[ckan harvester] Support for default tags and groups
2011-11-18 13:20:41 +00:00
8ec05bc3e3
Modify import command to avoid problems with the Session
2011-11-15 11:26:24 +00:00
c04d80e27e
Use get_action function instead of directly calling the action functions
2011-10-26 17:26:18 +01:00
31dac7029a
fix to make import stage work on its own
2011-09-28 14:27:28 +01:00
63afd199a9
Add link to source documents from object errors
2011-09-08 10:27:36 +01:00
31c1ea1c21
Separate gather and object errors in reports. Add info about guid and object id in the object ones
2011-09-08 09:58:21 +01:00
c36d9bdd8e
Add new command to create new jobs for all active sources
2011-09-06 18:25:17 +01:00
dd00e98d9d
[model]: More careful about creating tables, since paster db upgrade loads the envrionment and therefore runs setup() before it does the migrations, and therefore in this instance we do not want to create the db tables.
2011-08-10 16:25:57 +01:00
7927329536
Make harvesters work with latest ckan release
2011-07-29 11:31:03 +01:00
bbf623213e
Added tag ckanext-harvest-0.1 for changeset 9bc6d9d11f17
2011-07-29 10:47:55 +01:00
fda4534994
Remove old tests
2011-07-19 14:18:25 +01:00
cabbb4922d
Use API version defined in config if present
2011-07-18 17:35:32 +01:00
c867660e7d
Add docs to base harvester functions
2011-07-18 17:35:03 +01:00
98384021fc
Update README. Thanks to Rolf Kleef for the patch
2011-07-18 17:34:24 +01:00
54de6759fe
Fix bug with empty config
2011-06-28 15:04:40 +01:00
c80e68a12f
Ensure the correct configuration is used on each stage
2011-06-14 15:59:13 +01:00
3125bb1514
Add a check to ensure sources with no packages are reharvested
2011-06-14 12:59:48 +01:00
2ac9885150
Page packages in the harvest source details page
2011-06-14 10:27:48 +01:00
0d9d1f8096
reduce number of queries for harvest index to a less insane number. still heavy.
2011-06-13 17:36:35 +02:00
ef04ce1774
Add support for config options in CLI
2011-06-13 15:56:19 +01:00
13f2fb3b96
use hasattr for config validation
2011-06-09 11:35:58 +02:00
7b61fb62bf
Ignore missing config values
2011-06-07 15:32:46 +01:00
98bfd50f47
Load config in the CKAN harvester
2011-06-07 13:35:11 +01:00
6e75d362e3
Add a simple way for harvesters to store configuration options. If form_config_interface is Text on the info dictionary, the configuration field will be enabled in the form. Harvesters can also provide a validate_config method.
2011-06-07 12:07:53 +01:00
ca6af0249a
Reverting previous changeset, as it conflicts with dgu_form_api
2011-06-07 11:58:35 +01:00
f9c0ee37aa
spacing in template paths
2011-06-06 10:16:34 +02:00
89934b8538
[harvesters] factor out a base harvester for use in generic harvesting apps
2011-06-02 12:07:07 +02:00
79fd966573
add database setup at configure time
2011-05-31 18:06:26 +01:00
264b606c48
take tables out of global scope at import time
2011-05-31 18:02:07 +01:00
amercader
David Read
David Raznick
Friedrich Lindenberg