harvester-d4science/CHANGELOG.rst

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog and this project adheres to Semantic Versioning

********* 1.5.6_ - 2023-06-26*********

Fixed

• Fix url endpoint for job_show #534

********* 1.5.5_ - 2023-06-05*********

Fixed

• Fix display of harvest job errors #533

********* 1.5.4_ - 2023-05-23*********

Fixed

• Fix a problem with data-dictization when using sqlalchemy 1.4+ #529

********* 1.5.3_ - 2023-04-03*********

Fixed

• Fix asset path in MANIFEST.in #525

********* 1.5.2_ - 2023-03-28*********

Fixed

• Fix URL endpoints: from harvest.object_show to harvester.object_show #524

********* 1.5.1_ - 2023-03-22*********

Fixed

• Fix url_for routing to point to harvester blueprint #523

********* 1.5.0_ - 2023-03-16*********

Changed

• Added unescape for email text body to avoid encoded characters #517
• Pick the right harvest_object_id if there are multiple #519
• Do not duplicate harvest_extras if exist in root schema #521
• Use 403 when actions are forbidden, not 401 #522
• Drop support old versions #520

Breaking Changes -------- h.bootstrap_version() no longer exist since it is no longer needed to inject CSS classes - Support for old Pylon's route syntax has been removed. Example: calling url_for("harvest_read") will no longer work. URLs for ckanext-harvest needs to respect Flask's syntax: url_for("harvest.read"), etc

********* 1.4.2_ - 2023-01-12*********

Changed

• Add DB index harvest_error_harvest_object_id_idx #514
• Remove pyopenssl requirement c87309a
• Add CSRF protection to new source form #516

********* 1.4.1_ - 2022-09-20*********

Changed

• Use requirements.txt instead of pip-requirements.txt (still working via symlink) 8ed1eca

Fixed

• Bump pyopenssl requirement to avoid requirements error on install 98edcd3
• Fixes unicode error in Python 2 #502
• Fixes in email notification sendngi #499, #505
• Fix pagination for Dataset list on source page #504

********* 1.4.0_ - 2022-04-20*********

Changed

• Add ckan.harvest.not_overwrite_fields #472
• Support for Bootstrap 5 templates #490
• Support for CKAN 2.10 #492 #496

Fixed

• Fix JSONDecode error #489
• Check if email exists before sending notification #498

********* 1.3.4_ - 2022-01-24*********

Changed

• Changes function calls to render_jinja2 over to render as the former is no longer used. #459
• Set the default value for MQ_TYPE to redis #463
• Add option keep-current to clearsource_history command #484

Fixed

• Fix JSON serialization for Python3 #450
• Make Rehavest and Clear buttons work again #452
• Fix error when running run-test #466
• Fix timeout calculation #482
• Fix harvest extras for packages #458

********* 1.3.3_ - 2021-03-26*********

Changed

• Migrate tests from Travis CI to GitHub Actions
• Optimize last error free job detection #437

Fixed

• Improve timeout detection #431
• Check if Redis key is available #432
• Include webassets.yml in MANIFEST

********* 1.3.2_ - 2020-10-08*********

Changed

• Calculate timeouts based on last finished object instead of job creation time #418

Fixed

• Fix resubmitting harvest objects to Redis fetch queue #421

********* 1.3.1_ - 2020-09-01*********

Changed

• Abort failed jobs CLI command #398

Fixed

• Fix Redis conflict with core workers
• Fix harvest source list reference
• Fix and improve test suite, remove nose tests

********* 1.3.0_ - 2020-06-04*********

Changed

• Support for Python 3 #392
• Add option for job timeout #403
• Add support for limiting number of results and filtering by organization in harvest_source_list #403

Fixed

• Fix support for different Redis client libraries #403
• Fix force_import option in run_test command #402
• Fix show object #395
• Fix handling of exceptions in controller #390

********* 1.2.1_ - 2020-01-22*********

Changed

• Support not modified status for objects #385
• New force-import flag for the run_test command #385

Fixed

• Get message from harvest_object_error-dict #381
• Fix Admin link appearing to non authorized users #389
• Capture Redis Exceptions #385

1.2.0 - 2019-11-01

Changed

• Apply flake8 to be PEP-8 compliant #354
• Use ckantoolkit to clean up imports #358
• Add hook to extend the package dict in CKAN harvester
• Use CKAN core ckan.redis.url setting if present

• Remove database migration code targeting ancient versions #376

  (In the unlikely event that you need to upgrade from one

  of the previous DB versions just apply the changes removed on the linked PR manually)

Fixed

• harvest_source_type_exists validator should not fail if Harvester has no info() method #338
• Fix SSL problems for old versions of Python 2.7.x #344
• Add an 'owner_org' to the v3 package migration #348
• Fix harvest request exceptions #357
• Fix wrong toolkit reference 8e862c8
• Mark early errored jobs as finished 5ad6d86
• Resubmit awaiting objects in the DB not on Redis 5ffe6d4

1.1.4 - 2018-10-26

Fixed

• Fix nav link

1.1.3 - 2018-10-26

Fixed

• Reduce usage of c vars (CKAN 2.9)

1.1.2 - 2018-10-25

Added

• Send harvest-error-mails to organization-admins #329
• CKAN Harvester option to include/exclude groups #323
• Use Redis password from configuration when present #332
• Support for CKAN 2.9

Fixed

• Ensures the AND operator for fq in solr #335
• Fix styling issues on Bootstrap 3

1.1.1 - 2018-06-13

Added

• Move CKANHarvester._last_error_free_job to HarvesterBase.last_error_free_job #305
• Add the CSS classes for FontAwesome 4.x #313
• Add config option for dataset name append type #327
• Send error mail to admin when harvesting fails #244

Changed

• Readme test tip ckan parameter #318

Fixed

• Fix handling of clean_tags options for tag lists and dicts #304
• Don't delete all solr documents/fail to index harvesters when harvest config blank #315
• Fix print statements to be Py3 friendly #328

1.1.0 - 2017-11-07

Added

• Button on harvest admin page to abort running jobs #296

Changed

• Test improvements for harvester config #288
• Use package_search API for count of datasets #298
• Catch sqlalchemy.exc.DatabaseError instead of sqlalchemy.exc.OperationalError in gather_callback #301

Fixed

• Fix default_extras initialization #290
• Travis build (postgres service, checkout of correct CKAN branch, libcommons-fileupload) #297

1.0.0 - 2017-03-30

Added

• Includes i18n directory in package.
• Adds a new clearsource_history command/operation.
• Adds new parameter return_last_job_status to harvest_source_list
• Documentation for logs API

Changed

• gather_stage return empty list instead of None if errors occured
• Change redirect calls to h.redirect_to

Fixed

• Fix namespace package declarations
• Only purge own data when calling queue_purge with redis
• Fix default_groups behavior

0.0.5 - 2016-05-23

Added

• Adds HarvestLog to log to database
• Adds a new clean_harvest_log command to clean the log table

Removed

• This release removes support for CKAN <= 2.0

0.0.4 - 2015-12-11

Added

• Adds _find_existing_package method to allow harvesters extending the HarvesterBase to implement their own logic to find an existing package
• Adds support for ITranslation interface
• Adds special CSS class to datetimes in frontend to enable localisation to the users timezone

Changed

• Make statistics keys consistent across all actions

Removed

• Remove harvest_source_for_a_dataset action

0.0.3 - 2015-11-20

Fixed

• Fixed queues tests

0.0.2 - 2015-11-20

Changed

• Namespace redis keys to avoid conflicts between CKAN instances

0.0.1 - 2015-11-20

Added

• Adds clear source as a command
• Adds specific exceptions instead of having only the generic Exception

Fixed

• Catch 'no harvest job' exception

Categories

• Added for new features.
• Changed for changes in existing functionality.
• Deprecated for once-stable features removed in upcoming releases.
• Removed for deprecated features removed in this release.
• Fixed for any bug fixes.
• Security to invite users to upgrade in case of vulnerabilities.