Catalogue Util Library

A comprehensive Java utility library for interacting with D4Science's Data Catalogues, providing seamless integration with CKAN-based data repositories.

Table of contents

• Overview
• Key Features
• Main Components
• Usage Examples
• Built With
• Requirements
• Integration
• Documentation
• Changelog
• Authors
• How to Cite
• License
• About gCube
• Funding

Overview

This library serves as a unified interface to perform both read and write operations on D4Science data catalogues, abstracting the complexity of direct API interactions and providing higher-level functionality for catalogue management.

Key Features

• Dual Operation Support:

  • Read operations (datasets, users, organizations, resources) powered by enhanced jackan client library
  • Write operations (create/update datasets, resources, metadata) powered by gCat service

• Configuration Management: Automatic discovery and configuration of catalogue endpoints through gCube context

• Content Moderation: Built-in support for content moderation workflows with approval/rejection mechanisms

• Multi-Organization Support: Handle multiple organizations and user roles within the same context

• Database Integration: Direct CKAN database access for advanced queries and bulk operations

• Extended CKAN Client: Enhanced timeout handling and custom request configuration

Main Components

Core Classes

• DataCatalogueService: Main entry point providing comprehensive catalogue operations
• GCatCaller: Interface to gCat service for write operations and configuration management
• ExtendCkanClient: Enhanced CKAN client with custom timeout and request handling
• DataCatalogueFactory: Factory pattern for creating service instances with proper scope management
• CatalogueContentModeratorSystem: Content moderation and approval workflow management

Configuration & Database

• GcatConfiguration: Bean for managing gCat service configuration (URLs, tokens, database settings)
• DataCatalogueFrontEndConfig: Frontend-specific configuration management
• DBCaller: Direct database access for CKAN PostgreSQL backend
• CkanDB: Database connection configuration and utilities

Usage Examples

Basic Setup

// Get service instance for a specific scope
DataCatalogueFactory factory = DataCatalogueFactory.getFactory();
DataCatalogueService service = factory.getUtilsPerScope("/gcube/devsec/devVRE");

// Access catalogue URL and basic info
String catalogueURL = service.getCatalogueUrl();
String portletURL = service.getPortletUrl();

Read Operations

// Get dataset information
CkanDataset dataset = service.getDataset("dataset-name", "user-api-key");

// List organizations
List<CkanOrganization> organizations = service.getOrganizations();

// Get user information
CkanUser user = service.getUserFromApiKey("user-api-key");

Write Operations

// Create a new dataset
DatasetBean datasetBean = new DatasetBean();
datasetBean.setTitle("My Dataset");
datasetBean.setName("my-dataset");
// ... set other properties

String datasetId = service.createDataset(datasetBean);

Content Moderation

// Get content moderation system
CatalogueContentModeratorSystem cms = service.getCatalogueContentModerator();

// Check if moderation is enabled
boolean moderationEnabled = cms.isModerationEnabled(false);

// Get items with specific status
List<String> pendingItems = cms.getListOfItemsByStatus(ItemStatus.PENDING, 10, 0);

Built With

• OpenJDK 8+ - The JDK used
• Maven - Dependency Management
• CKAN API - Data catalogue platform
• gCat Service - gCube catalogue service
• Jackson - JSON processing
• PostgreSQL - Database backend

Requirements

• Java 8 or higher
• Maven 3.x
• Access to a gCube infrastructure with catalogue services enabled
• Valid gCube security tokens for authentication

Integration

Add this dependency to your pom.xml:

<dependency>
    <groupId>org.gcube.datacatalogue</groupId>
    <artifactId>catalogue-util-library</artifactId>
    <version>[2.0.0-SNAPSHOT, 3.0.0-SNAPSHOT)</version>
</dependency>

Documentation

You can find the D4Science Catalogue documentation at GCat Background Wiki Page

Changelog

See CHANGELOG.md.

Authors

• Francesco Mangiacrapa (ORCID) Istituto di Scienza e Tecnologie dell'Informazione (ISTI), Consiglio Nazionale delle Ricerche (CNR), Pisa, Italy
• D4Science Infrastructure, Pisa, Italy

How to Cite

If you use this software, please cite it using the metadata in CITATION.cff

License

This project is licensed under the terms specified in the LICENSE.md file.

About gCube

This software is part of the gCube Framework: an open-source software toolkit used for building and operating Hybrid Data Infrastructures enabling the dynamic deployment of Virtual Research Environments by favouring the realisation of reuse oriented policies. The gCube Framework supports the development and operation of the D4Science Infrastructure.

Funding

See FUNDING.md