bbe459527f
[forms] Major refactoring of the harvest forms. Forms no longer use the DGU form
...
API, and are handled similarly to the new ones on CKAN core (logic, schema,
validators...). The UI is also more consistent with the CKAN one.
2011-05-13 14:17:58 +01:00
26cdc1089d
Change date definitions in Harvest Objects. reference_date -> metadata_modified_date, created -> gathered
2011-05-11 17:07:05 +01:00
e1080e349e
Set a flag to force harvesters to import objects
2011-05-10 17:11:12 +01:00
e320d0588f
Add command to reimport existing harvest objects
2011-05-10 16:06:57 +01:00
f7c6854a1d
Save reference date in Harvest Objects when harvesting CKAN instances
2011-05-10 12:57:57 +01:00
329ca2dd29
Add a reference date to the Harvest Objects. This must be set during the harvest
...
process.
2011-05-10 11:05:44 +01:00
c697bc3350
Normalize https ports too ( #736 )
2011-05-09 18:47:30 +01:00
5594f22be7
More robust URL checking ( #736 )
2011-05-09 14:03:46 +01:00
0e56c0ab4f
Abort pending jobs when removing sources
2011-05-05 17:13:07 +01:00
e0a1e5752d
Fix redirects in view controller
2011-05-05 16:47:34 +01:00
35fdfaf58c
[pip-requirements] Add carrot dependency.
2011-04-28 11:45:57 +02:00
36b93359fa
[merge] from 987-common-harvesting-framework.
2011-04-28 11:14:13 +02:00
43453b6938
Show GUID on object errors
2011-04-19 17:16:25 +01:00
c9d43b2e4d
overide default create schema
2011-04-19 15:34:56 +01:00
e3bca3ceee
Add first version of the CKAN harvester [ #985 ]
2011-04-19 14:54:59 +01:00
b5aea2d863
Allow sources with existing URLs if the existing ones are inactive
2011-04-18 17:19:04 +01:00
bc8caebc30
[packaging] Moved templates and publics into source
2011-04-18 08:49:25 +00:00
7e75673cff
Add function to edit harvest sources
2011-04-15 18:28:58 +01:00
e3a83ea14d
Source URL not unique
2011-04-15 18:28:38 +01:00
1edf984f1b
Log an error if no harvester could be found for the source
2011-04-15 16:12:47 +01:00
d92b1df8c6
Only send harvesting jobs from active sources to queue
2011-04-15 15:50:32 +01:00
f492ed9c7a
Update README with harvesting interface description and instructions for running the jobs
2011-04-15 15:36:53 +01:00
5fb6512d82
Do not output messages when running the 'run' command
2011-04-15 15:35:19 +01:00
9018beeedb
use create_all instead of create
2011-04-15 12:19:24 +01:00
a11480aa8f
[refactoring] Use ckan.harvest for options in the ini file
2011-04-13 12:46:52 +01:00
040e4d6956
[refactoring] Add a command to create the necessary tables in the database
2011-04-13 12:39:53 +01:00
312e9b8209
[refactoring] Update view controller and templates
2011-04-12 16:15:14 +01:00
aaf7885f43
[refactoring] Return default values if provided in getters. Also return dates as strings
2011-04-12 16:13:38 +01:00
a234d6ff6a
[refactoring] Remove unneeded interface
2011-04-12 10:11:02 +01:00
842bea8433
[refactoring] Remove all geo related functions as they now have it own extension ckanext-spatial
2011-04-11 19:19:09 +01:00
280dce1626
[refactoring] Add status info to harvest sources. Updated dict functions and UI.
2011-04-11 16:30:56 +01:00
e2faa14b8d
[refactoring] Save gathering starting and finishing time
2011-04-11 16:29:47 +01:00
3c505693ff
[refactoring] Do not delete sources, just inactivate them. Also don't delete jobs.
2011-04-08 17:07:19 +01:00
2588352bc5
[refactoring] Simplify model relations
2011-04-08 16:48:29 +01:00
9465d59d3a
[refactoring] Remove old harvesting controller. This functionality now lives on ckanext-inspire
2011-04-08 15:54:33 +01:00
b38fc57ec5
[refactoring] Changes in the queue code and the IHarvester interface"
2011-04-07 16:59:11 +01:00
90ae9d27db
[refactoring] Tweaks in the model and a setup function to create the tables
2011-04-07 16:57:36 +01:00
56d5acc867
[refactoring] Add code to handle queuing and the new IHarvester interface. Add
...
new commands in the CLI to start the queue consumers and fire the harvesting
process.
2011-04-06 12:45:00 +01:00
f3c84a6dba
mention requirements explicitly in setup.py
2011-04-05 22:39:33 +01:00
adbadffcdf
merge
2011-04-05 22:32:36 +01:00
3ce7cf2dc5
documentation tweak
2011-04-05 22:31:49 +01:00
4023bb7222
[refactoring] Use the common functions in the web interface.
...
Not yet implemented in create and edit harvest source as they use the DGU forms API.
Also TODO, think of what report info is needed in the listing and details page.
2011-04-05 13:39:23 +01:00
3d32a18802
[refactoring] Return dictionaries from the common functions, and use them in the CLI"
2011-04-05 12:55:58 +01:00
dca4ff328f
[refactoring] Add missing column in error table
2011-04-05 12:53:36 +01:00
a4d93e4ad3
[refactoring] Add a reference to objects from jobs
2011-04-05 12:36:27 +01:00
e819a68f21
[refactoring] Move common functions to lib. Adapt the CLI to use these common functions.
2011-04-05 11:53:39 +01:00
ce86cfde1b
[refactoring] Clean up the model definitions
2011-04-05 11:51:59 +01:00
4220f67507
Move Gemini tests frm ckan core to ckanext-harvest ( #1030 )
2011-04-04 14:34:41 +01:00
65c81a3014
Add more information in save extent exceptions
2011-04-04 14:31:54 +01:00
d3f06b7b4f
[refactoring] Changes in model
2011-03-30 16:55:42 +01:00
Adrià Mercader
David Read
David Raznick
james
ww