Commit Graph

699 Commits

Author SHA1 Message Date
David Read 8a7bc9e1d8 Merge remote-tracking branch 'origin/master' into immediate-harvest
Conflicts:
	README.rst
	ckanext/harvest/commands/harvester.py
	ckanext/harvest/logic/action/create.py
	ckanext/harvest/logic/action/update.py
	ckanext/harvest/logic/auth/update.py
2015-11-03 00:40:25 +00:00
amercader 3f856c3b8c Merge branch 'activate-pyenv' 2015-10-30 14:12:56 +00:00
amercader 975db5a6c9 Merge branch 'harvest-command' 2015-10-30 14:12:50 +00:00
David Read f1d2d5fdc4 [#111] Run jobs straight away. 2015-10-28 21:58:36 +00:00
David Read a617979338 [#141] Add install instruction to activate the pyenv. Fixes #141. 2015-10-28 20:05:43 +00:00
David Read 421e6da660 Add run_test, job_abort, source commands
* run_test - for running a whole harvest on the command-line
* job_abort - for aborting a limbo job
* source - for showing a single harvest source
* allowing a source to be specified by name in several commands
2015-10-28 17:51:58 +00:00
amercader 86630adab7 Merge branch 'include-exclude-org' 2015-10-27 15:52:55 +00:00
amercader 24574f485b Setup harvest model in harvester tests 2015-10-23 15:43:01 +01:00
David Read bc49149d5e Merge branch 'master' into include-exclude-org 2015-10-23 14:36:53 +01:00
amercader 2f4adfb338 Merge branch 'tests' 2015-10-23 13:18:15 +01:00
amercader 992cdc57d2 Test 2.4 and latest branches for all releases 2015-10-23 11:56:12 +01:00
amercader 1bad80e273 Merge branch 'purge' 2015-10-23 11:52:35 +01:00
amercader 3c6cc55be0 Only flush keys on the current Redis database 2015-10-23 11:52:22 +01:00
amercader fdbade465f Merge branch 'master' into purge 2015-10-23 11:33:43 +01:00
amercader d950b13400 Merge branch 'unique-names-improved' 2015-10-23 11:02:49 +01:00
amercader 9b10f9e227 Merge branch 'migration-states' 2015-10-23 10:59:17 +01:00
amercader 501edffe2d Merge branch 'master' into migration-states 2015-10-23 10:59:04 +01:00
amercader 0f12c71aef Merge branch 'docs' 2015-10-23 10:56:23 +01:00
David Read 5ec47db63e Merge pull request #166 from LondonAppDev/master
Fixed Typos.
2015-10-21 17:57:02 +01:00
David Read 3e4a9933ce Remove prints. 2015-10-21 16:52:19 +00:00
Mark Winterbottom 6fb5728d02 Fixed Typos. 2015-10-21 17:48:07 +01:00
David Read dc7af5d150 Remove prints. 2015-10-21 16:38:03 +00:00
David Read eb9aa17862 Include/exclude orgs funcationality based on work by memaldi and ross. 2015-10-21 16:33:16 +00:00
David Read f70c16bce7 Add framework for testing harvesters. Modernize existing tests. 2015-10-21 16:26:57 +00:00
David Read d1f84295f8 purge_queues command now has warning about impact of Redis flushall, plus add some (log) output when you run a purge. 2015-10-21 16:12:40 +00:00
David Read 1f81fefcbb Correct doc error for configuring amqp. 2015-10-21 16:11:11 +00:00
David Read 6360681a8f [#105] Fix order of deletes, as agreed with @florianm. 2015-10-12 15:57:27 +01:00
David Read 372fc56c89 Merge branch '105-clean-the-cleaner' of https://github.com/datawagovau/ckanext-harvest into datawagovau-105-clean-the-cleaner 2015-10-09 15:00:56 +01:00
David Read 82bdff2f34 Add tests 2015-10-01 17:59:17 +01:00
David Read be3e88086a Generating unique names improved
* Harvesters that change the name when the title changes have had a
  problem when the change is small and a number was unnecessarily
  appended. e.g. "Trees "->"Trees" meant _gen_new_name("Trees") returned
  "trees1". Now you can specify the existing value and it will return
  that if it still holds.
* Maximum dataset name length is now adhered to.
* To make a name unique, a sequential number is now added, since for
  users that is more understandable and pleasant. However hex digits are
  still an option, for those that want to harvest concurrently.
2015-10-01 17:53:03 +01:00
Adrià Mercader 89b6ad2ce1 Merge pull request #150 from raphaelstolt/templates_new-fix
[#103] Include templates_new directory
2015-09-09 11:32:57 +01:00
Raphael Stolt d9fc9bdfc6 [#103] Include fanstatic styles directory 2015-08-19 14:58:16 +02:00
Florian Mayer a6cdda0a14 set max version to 2.4.99 2015-08-19 08:41:42 +00:00
florianm 1905caa961 upgrade harvest_source_clear to not delete from authz models removed in migration 078 2015-08-19 10:25:20 +08:00
Raphael Stolt a77b62c25a [#103] Include templates_new directory 2015-08-17 12:11:45 +02:00
David Read 46f7b32b04 Merge branch 'master' of github.com:okfn/ckanext-harvest into migration-states 2015-07-22 10:13:55 +01:00
David Read 2da918c2e4 Fix migration for old harvests so that ones that errored are correctly marked. Added helpful comments in model. 2015-07-22 10:13:02 +01:00
amercader 83cca925c9 Merge branch 'metaodi-145-catch-no-harvesting-jobs-exception' 2015-07-21 11:16:43 +01:00
Stefan Oderbolz ab76830e85 [#145] Throw + catch a custom exception if there are no jobs to run
If there are no harvesting jobs to run, there was always an ugly
exception message when using the paster command. This replaces the ugly
output with a proper message and uses a custom exception to allow others
to deal with this error differently.
2015-07-20 18:41:50 +02:00
David Read 5801f3e083 Merge pull request #142 from metaodi/138-timezone-display
[WIP] [#138] Add data attributes to support timezone conversion
2015-07-09 22:03:18 +01:00
Stefan Oderbolz 83dd0b4b68 [#138] Add data attributes to support timezone conversion 2015-07-09 22:35:54 +02:00
David Read ce0aedd5b9 Merge pull request #140 from metaodi/139-delete-package-relationships
[#139] Delete package relationships when clearing a harvest source
2015-06-26 16:22:37 +01:00
Stefan Oderbolz 4dc2f7367d [#139] Delete package relationships when clearing a harvest source 2015-06-26 17:20:23 +02:00
amercader 955d3bb84b Merge branch 'tryggvib-master' 2015-06-15 17:40:04 +01:00
Tryggvi Bjorgvinsson 7a8fc2c67e Add Swedish translations
Harvester has been translated into Swedish. This updates the .po
file from transifex and a resulting compiled .mo file. Thanks to
translator Börje Lewin.
2015-06-15 14:28:20 +00:00
amercader 88d9ba0397 [#136] Fix broken RabbitMQ queue names
The harvester command was still using the old ones.
Use specific ones for testing.
2015-06-11 13:56:22 +01:00
amercader 673dfc9882 [#127] Use site user on the CKAN harvester
Add missing call
2015-06-11 10:38:33 +01:00
amercader d3a3f09ad1 [#127] Use site user on the CKAN harvester
To avoid having to create a 'harvest' sysadmin explicitly. It will still
be used if present, but if not the site user will be used. You can also
define to user to use via a config option.
2015-06-11 10:19:07 +01:00
amercader 84126bb821 Update README
Simplify install instructions (no need to choose a branch on
ckanext-harvest, just use master). Updated examples and paths.

Added contributing and license info.
2015-06-10 15:57:28 +01:00
amercader 5b4bff329f Merge branch 'metaodi-clear-command' 2015-06-10 15:34:13 +01:00