ckanext-d4science_theme

The CKAN extension that implements the D4Science theme template used by D4Science Catalogues.
The most important features are the possibility to save dataset with duplicated keys in the extra fields section and an improved managment of datasets visibility.

Table of Contents

• Requirements
• Installation
• Major Changes from 2.6 to 2.10
  • Routing changes
  • Privatedatasets integration
  • Duplicated extras keys
  • List of other plugins that needed to be customized
• Updated User Interface
• Styling
• Applying SCSS Changes
• Images and Logos
• Custom Icons for Types
• Customizing access to dataset's resources
• Enhancements
  • Spatial Query Lazy Loading Component
    • Overview
    • Purpose
    • Key Features
    • Dependencies
    • URL Parameters

Requirements

None

Installation

To install the ckanext-d4science_theme, use the docker-ckan repository, which includes this extension by default.

Major Changes from 2.6 to 2.10

This plugin migrate from Ckan version 2.6 to Ckan 2.10.5.
To do so, the base infrastructure of the plugin had to change:

• The code was updated from Python2 to Python3;
• Pylons framework got changed with Flask.

The plugin privatedataset was updated and inclued too.

Changes in details

As the first step, the code was updated using the automatic python command 2to3 that converted each file from python2 syntax to python3.
Then, each function that was listed as deprecated by the ckan official CHANGELOG, got removed and substitued with the corrisponding new one. For example the methods: before_create, before_view, etc...were changed into before_dataset_create, before_dataset_view, etc...

Routing changes

Ckan changed its routing structure from Pylons to Flask. Pylons uses controllers to manage the view and routing of the ckan application, Flask uses Blueprints and rules to manage the views, instead.
The old "controller structure" has changed into "views structure".

Ckan 2.6

def before_map(self, map):  
  [...]
  map.connect('/organization_vre', controller='ckanext.d4science_theme.controllers.organization:OrganizationVREController', action='index')
  map.connect('/organization_vre/{id}', controller='ckanext.d4science_theme.controllers.organization:OrganizationVREController', action='read')
  [...]
  return map

#in controllers folder
class OrganizationVREController( toolkit.DefaultOrganizationForm, toolkit.DefaultGroupForm):

Ckan 2.10.5

def get_blueprint(self): 
  [...]
  for url, endpoint, view_func in rules:
    blueprint.add_url_rule(url, endpoint, view_func)

  blueprint.register_blueprint(organization_vre_blueprint)
        
  return blueprint

#in views folder
organization_vre_blueprint = Blueprint('organization_vre', __name__)

@organization_vre_blueprint.route('/organization_vre', strict_slashes=False)
def index():
  [...]

Privatedatasets integration

The plugin privatedatasets was originally used to make "searchable" datasets with the attribute Private set to true. So even if a user is not logged or is not a part of the dataset's organization can see the dataset in the list of all available datasets, but without the possibility to access it, unless he logs into the site. If a user doesn't have the permission to access the dataset a pop-up will be shown requesting the login. After CKAN 2.9 the method that retrives all the dataset has changed, making it even more robust. So in order to maintain this feature, a custom package_search method was developed. It works like this:

1. Ckan checks if the user is anonymous or he isn't part of the organization/group, if it is the case, the function removes the permission check from the context and add a facet to search all the datasets with the parameter searchable == True;

2. Then the "original" package_search from CKAN is called.

Duplicated extras keys

D4Science_Theme Plugin offers the possibility to save multiple keys in the extras field with the same name, but different values. This is useful if a dataset as more than one author for example.

The extras fields are also used to define custom type and give a better data visualization for every single dataset. This is done by using special attributes declared inside the custom extras field.

To make a type, the user has to write system:type as a key and the associated value will be the type. (that can be used as a facet in the package_search).

To make a table that merges some attributes inside the dataset detail page, the user must save all the extras fields with the same namespace, like this namespace:attribute_key.

List of other plugins that needed to be customized

D4science personalized other plugins, so they were updated to CKAN 2.10 too.

• DCat
• GeoNetwork
• Harvest
• Hierarchy
• Oaipmh
• Spatial

The customizations mainly focus on fixing harvesting functions: extras fields and searchable must be imported.

Updated User Interface

The look and feel of the catalog have been almost completely redesigned compared to the previous version.

Starting with the homepage, the search bar is now prominently featured, and a tabbed structure has been introduced to display various organizations, types, tags, and more.

Other important UI changes can be found in the dataset card within the Organizations section.

In addition to the title, description, and system type already present before, it is now possible to view the publication date and the author when the dataset is not private.

The most significant change, which deeply modifies CKAN’s default structure, has been implemented in the dataset detail page.

Typically, all catalog pages in CKAN follow a structure defined within the page.html file (in the templates folder). However, for the dataset detail page (templates/package), a new custom_page.html has been created, defining a different structure from the standard one by repositioning snippets in alternative locations.

The main modified files include:

• custom_page.html
• read_base.html
• /snippet/info.html
• and others.

Styling

The website’s styling is managed through SCSS files located in assets/scss.

Within this directory, the _variables.scss file defines the color variables used in d4science_theme.scss, where all the site styles are structured and categorized with section comments.

Applying SCSS Changes

To apply SCSS modifications to the site, use the VSCode extension "Live Sass Compiler" (formerly "Watch Sass") and activate it. This generates a CSS file from the SCSS source.

Through the webassets.yml file and its definition within base.html, both the stylesheets and JavaScript scripts located in assets/js/d4science_scripts.js are selected for use.

Images and Logos

Images, logos, and other visual assets are stored in the /public folder.

Custom Icons for Types

The plugin is able to set custom icons for every types in the catalogue. The are two approach to this. If both configuration are present, the plugin will always choose the image one.

Image approach

Populate the ckan.focused_dataset_types with pairs formed by <type:image_file_name> separeted by [,].

The files MUST be putted inside the /catalog/icons/tags-icons/ path.

Example:

CKAN__FOCUSED_DATASET_TYPES="dataset:database.svg, service:service_tag.svg, deliverable:papers-text.svg"

This line will associate the dataset type with database.svg, the same will happen to service and deliverable types.

Icons approach

Populate the ckan.focused_dataset_types_icons with pairs formed by <type:icon_class> separeted by [,].

The icons can be found on Font Awesome: you need to copy the only the "class" value.

Example:

CKAN__FOCUSED_DATASET_TYPES_ICONS="dataset:fa-solid fa-database, deliverable:fa-solid fa-newspaper"

This line will associate the dataset type with a database icon, the same will happen to the deliverable type with a newspaper icon.

Customizing access to dataset's resources

The D4Science plugin allows an organization to grant access to all of its resources or for resources of specific types of datasets for external/anonymous users.

If no configuration is provided, the default policy is: all denied.

The levels of permission are structured as follows:

• organization level: all the resources of every type of dataset can be access or not.
• dataset type level: all the resource of the specified datasets can be access or not. If an organization has other types of dataset not specified in the production.ini the default policy will be applied for them. For example: if an organization allows the access to all the resources of the "type1" datasets and doesn't specified anything else, the access to the resources of the "type2" datasets will be denied.

In order to customize this feature, the user has to edit the json of CKANEXT__D4SCIENCE_THEME__PUBLIC_DATASET__PUBLIC_ACCESS_TO_RESOURCES variable.

Pay attention to spaces, also if both level are specified for a single organization the dataset type level will be used.

Example:

CKANEXT__D4SCIENCE_THEME__PUBLIC_DATASET__PUBLIC_ACCESS_TO_RESOURCES={"test2": True, "open_science.system_type.policy": True, "fao": False, "fao.system_type.F2DSItem": True}

From the example:

• the organization test2 grants access to all the resources of every type of its datasets;
• the organization open_science grants access to the resources of all the datasets with the policy type. The access for the resources of every others datasets with different type will be denied.
• the organization fao has definied both permissions, although the organization level is set to false, the plugin will grant access to the resources of the F2DSItem type datasets.

Enhancements

Spatial Query Lazy Loading Component

Overview

This template extends the standard spatial query functionality to provide lazy-loading capabilities for map widgets. It optimizes performance by deferring resource-intensive map initialization until actually needed, particularly when using Mapbox as a basemap provider.

Purpose

• Displays a spatial filter map widget for defining geographic search boundaries
• Implements lazy-loading for Mapbox-based maps to minimize API calls and costs
• Provides direct map display for non-Mapbox providers (e.g., OpenStreetMap)
• Maintains backward compatibility with existing spatial filtering features

Key Features

Lazy Loading Strategy

• Mapbox Integration: Map content loads via AJAX only when the user clicks "Show Spatial Filter" button
• Active Filter Detection: If ext_bbox parameter exists in URL, map loads immediately on page load
• Cost Optimization: Avoids unnecessary Mapbox API initialization until user interaction

Dynamic Behavior

• Toggle button to show/hide spatial filter interface
• Automatic state preservation based on URL parameters
• AJAX-based snippet loading from custom endpoint
• Dynamic CKAN module re-initialization after content injection

Non-Mapbox Providers

• Direct map rendering without toggle functionality
• Suitable for free mapping providers (OpenStreetMap)
• Loads spatial query assets on page initialization

Dependencies

• CKAN spatial extension assets (JS/CSS)
• Custom endpoint: d4s.d4s_spatial.spatial_query_snippet
• CKAN module initialization system
• Helper function: h.d4science_is_spatial_map_mapbox()

URL Parameters

• ext_bbox: Indicates active spatial filter; triggers immediate map display

Author

• Francesco Mangiacrapa (ORCID) Computer Scientist at ISTI-CNR Infrascience Group

This approach significantly improves the user experience by preventing unnecessary spatial computations during page initialization.