CKAN for GDI - User Portal

1. Overview

This project has two purposes:

1. Prepare a docker image for CKAN, with all extensions and configurations needed to deploy the catalogue for the User Portal.
2. Setup a development environment for testing, fixing and enhancing our customised CKAN and its extensions.

The CKAN images used are from the official CKAN ckan-docker repo.

The non-CKAN images are as follows:

• PostgreSQL: Official PostgreSQL image. Database files are stored in a named volume.
• Solr: CKAN's pre-configured Solr image. Index data is stored in a named volume.
• Redis: standard Redis image
• NGINX: latest stable nginx image that includes SSL and Non-SSL endpoints
• National Catalogue Mock: Python script to expose a synthetic RDF file, to test the Harvester.

CKAN and all the components are configured using environment variables that you can set in the .env file.

2. Before running locally

• Install docker with support to docker compose v2.
  • Ensure you have enough computer resources, if you are using colima: colima start --arch aarch64 --vm-type=vz --mount-type=virtiofs --vz-rosetta --cpu 4 --memory 10
• Ensure git submodules are pulled correctly after clone gdi-userportal-ckan-docker git submodule update --init
• If you are using an Apple Silicon laptop (e.g., M1 or M2), add the following property to your docker-compose.yml file within ckan-dev service to ensure compatibility:

services:

  ckan-dev:
    platform: linux/amd64
    build:
      context: ckan/
      dockerfile: Dockerfile.dev

3. Useful commands

3.1. Build and start dev environment

Use this mode if you are making code changes to CKAN and either creating new extensions or making code changes to existing extensions. This mode also uses the .env file for config options.

To develop local extensions use the docker-compose.yml file with help from the scripts under bin:

To build the images:

	bin/compose build

To install extensions from the src directory:

	bin/install_src

To start the containers:

	bin/compose up

Shut down and restart the whole ckan-dev container (use bin/compose up -d instead to reload new values from .env)

	bin/compose restart

3.2. Remove images and volumes

  bin/compose down -v

3.3. Logs

  bin/compose logs -f

3.4. Re-index SOLR

ckan -c ckan.ini search-index rebuild

4. Accessing the running instance

The ckan instance runs on port defined in CKAN_PORT_HOST (5500). Access it at http://localhost:5500.
A sysadmin account is created by default with the following credentials (defined in .env):

• CKAN_SYSADMIN_NAME=ckan_admin
• CKAN_SYSADMIN_PASSWORD=test1234

5. installing new extensions

The current agreement is:

• Fork all the branches from the extension's repo into GitHub GDI.
• Create and push a new branch following this standard: user-portal-{original branch name/tag name} Reminder, prefer always to use a release/tag branch, over main/master branches, to avoid not tested changes.
• Add a git submodule: git submodule add {git_ssh_url} src/ckanext-{extension}.
• Add branch reference into .gitmodules, if not added.
• Configure the extension via environment variables or via a setup script. Setup scripts must be added into ckan/docker-entrypoint.d/.
• Ensure the extension is working as expected.
• Update ckan/Dockerfile to install the new extension via pip.

All Docker Compose commands in this README will use the V2 version of Compose ie: docker compose. The older version (V1) used the docker-compose command. Please see Docker Compose for more information.

Copying and License

This work is licensed under multiple licenses. Here is a brief summary as of January 2024:

• All original source code is licensed under AGPL-3.0-only, copyright (c) 2006-2024 Open Knowledge Foundation and contributors.
• All documentation is licensed under CC-BY-4.0.
• github actions workflows are licensed under Apache-2.0.
• For more accurate information, check the individual files.