Commit Graph

1274 Commits

Author SHA1 Message Date
Adrià Mercader 6222e38c85
Merge pull request #431 from avdata99/fix_timeout
Fix last harvest time to avoid timeouts
2021-02-04 12:42:50 +01:00
Andres Vazquez eb246439c6 Fix last harvest time to avoid timeouts 2021-02-03 11:29:07 -03:00
amercader d00553831e Replace default path to CKAN core config file with the one on the container 2020-12-16 11:17:46 +01:00
amercader 95e82c222b Remove test-ga.ini, use env instead 2020-12-15 16:01:37 +01:00
amercader 2cb3ab3d81 Show badge just for master 2020-12-15 12:38:08 +01:00
amercader 76c011809d Migrate tests from Travis CI to GitHub Actions
After Travis stopped supporting Open Source projects we decided to move
to GitHub Actions, which has turned out to be excellent.

The workflow uses two jobs, one for basic syntax checking with Flake8
and a matrix of test run using the supported CKAN versions (not master
yet, more on this later). The tests run inside a Docker Container
pre-configured with a CKAN dev install (currently the
openknowledge/ckan-dev images are used). Additionally all services are
also Dockerized, using ckan/ckan-solr-dev and ckan/postgres-dev and the
standard redis image.
2020-12-15 12:31:24 +01:00
amercader edccfc4de0 Fix flaky redis test 2020-12-11 15:06:02 +01:00
amercader d3432e9c84 Merge branch 'travis-pg-again' 2020-11-24 13:45:09 +01:00
amercader 646a0603d8 Fix syntax 2020-11-24 10:41:09 +01:00
amercader 8365eb377c Test master allowing failures 2020-11-24 10:39:12 +01:00
amercader faed627f2d Use pg 12 on 2.9 2020-11-23 15:07:53 +01:00
amercader 762ee14402 Fix version check in templates to work on 2.10+ 2020-11-23 14:10:33 +01:00
Adrià Mercader 373822409d
Merge pull request #424 from andrecastro0o/423-Documentation-CKAN2.9
added CKAN 2.9 commands for harvesterext to README
2020-10-29 14:00:42 +01:00
Castro0o 6ab5ca4821 future proofing == 2.9 -> => 2.9 2020-10-29 12:03:28 +01:00
Castro0o 4badc3c0ea added CKAN 2.9 commands for harvesterext to README 2020-10-20 18:08:48 +02:00
amercader 8cde93c153 Tag version 1.3.2 2020-10-08 12:18:31 +02:00
amercader 7ad2bd5f38 Merge branch 'master' of github.com:ckan/ckanext-harvest 2020-10-08 12:16:48 +02:00
amercader ce97827ad7 Update changelog 2020-10-08 12:15:25 +02:00
Adrià Mercader 2f0a4b3ac1
Merge pull request #421 from ogdch/420-fix-check-for-harvest-objects-in-redis-queue
Fix checking for harvest objects in fetch queue.
2020-10-08 12:15:01 +02:00
Adrià Mercader 84f0f6b385
Merge pull request #418 from avdata99/417_improve_timeout_detection
Improve timeouts
2020-10-08 12:11:17 +02:00
Rae Knowler f34fb2fb51 Get harvest object ids out of redis in one go, instead of pop/push 2020-10-08 12:00:19 +02:00
Rae Knowler 7a9f9d9a82 Add fix to changelog 2020-10-07 15:43:38 +02:00
Rae Knowler 27f71a74c1 Fix checking for harvest objects in fetch queue.
The fetch queue is stored in redis as a list of harvest object ids. The previous code checked for individual harvest object ids in redis, and always found none. Therefore, every time the harvester run command was run, all ids of harvest objects in the WAITING state in the DB were re-added to the fetch queue.
2020-10-07 15:37:25 +02:00
Andres Vazquez a8fea5865f Improve error message 2020-10-02 11:42:09 -03:00
Andres Vazquez 39db3f89d7 Improve job times 2020-10-02 11:19:35 -03:00
Andres Vazquez 13a0525e3a Improve timeouts and add tests 2020-10-02 10:44:57 -03:00
Adrià Mercader 2be8842270
Merge pull request #416 from avdata99/415_change_email_renderer
Change email renderer
2020-10-01 15:33:24 +02:00
Andres Vazquez fc0df2f221 Change email renderer 2020-09-22 09:10:14 -03:00
Adrià Mercader dfd093aaa5
Merge pull request #407 from avdata99/406_notifications
Improve harvest jobs notifications
2020-09-17 15:59:19 +02:00
Andres Vazquez b798ded3d4 README for extensible actions 2020-09-14 11:12:31 -03:00
Andres Vazquez c84f63896c Create the chained action 2020-09-14 09:30:47 -03:00
Adrià Mercader d92864aa08
Merge pull request #414 from avdata99/av/upmaster
Indexes for harvest tables
2020-09-10 16:49:19 +02:00
Andres Vazquez 852a4d8ba6 Add index for harvest_object.guid 2020-09-08 09:40:58 -03:00
Andres Vazquez c6d53832cb Add an index for HarvetObjectExtra 2020-09-07 10:54:48 -03:00
Andres Vazquez 316a1fd251 Add more indexes to Harvest objects 2020-09-07 10:54:39 -03:00
Andres Vazquez ba9186015a Replace renderer 2020-09-01 10:06:01 -03:00
Andres Vazquez 644316ed17 Decrease logs verbosity 2020-09-01 07:49:29 -03:00
Andres Vazquez c19280facb Improve the IActions feature 2020-09-01 07:41:55 -03:00
Andres Vazquez 1e8465b3ba Define action to add notification recipients 2020-09-01 07:39:04 -03:00
Andres Vazquez 717804ce6d Clearify send emails calls and README 2020-09-01 07:39:04 -03:00
Andres Vazquez 719a37f8f4 Allow send notifications for all harvest jobs
Co-authored-by: Julie Kramer <thejuliekramer@gmail.com>
2020-09-01 07:39:04 -03:00
Andres Vazquez 762da5ebfa Add new config to send notifications for all harvest jobs
Add templates for email and function to setup variables on it
Update README
2020-09-01 07:36:55 -03:00
amercader df34890a2f Update to version 1.3.1 2020-09-01 11:50:50 +02:00
amercader da0e134853 Merge branch 'master' of github.com:ckan/ckanext-harvest 2020-09-01 11:50:37 +02:00
amercader bbff4e2a00 Merge branch 'mutantsan-abort-failed-jobs-cli-command' 2020-09-01 11:45:49 +02:00
amercader cb69ea556e Fix py3 ref 2020-09-01 11:39:09 +02:00
amercader 7ba9c6c6c1 Merge branch 'abort-failed-jobs-cli-command' of https://github.com/mutantsan/ckanext-harvest into mutantsan-abort-failed-jobs-cli-command 2020-09-01 11:28:26 +02:00
Adrià Mercader f0fb4f7b3f
Merge pull request #413 from bzar/fix-harvest_object_list-ref
Fix harvest_object_list member reference
2020-09-01 11:14:27 +02:00
amercader b8b0a3be05 Merge branch 'fix-redis-requirement' 2020-09-01 11:14:05 +02:00
amercader 8e79897206 Skip master tests until fanstatic is dealt with 2020-09-01 11:08:53 +02:00