Skip to content
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 62 additions & 26 deletions docs/geospatial-install.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,73 @@
# Setup Custom Executor Docker Image with the Geospatial Extension for KNIME Business Hub

Running a workflow on KNIME Business Hub requires an Execution Context (EC). An EC runs one or multiple pods. A pod holds Docker containers in turn, which run Docker images in the end. In general, one can simply use the official Docker images from the public [KNIME Artifact Registry](https://registry.hub.knime.com/) to create an EC.
## Introduction
Running a workflow on KNIME Business Hub requires an Execution Context (EC). Inside that EC runs one or multiple pods - that hold Docker containers - which run Docker images. In general, one can simply use the official Docker images from the public [KNIME Artifact Registry](https://registry.hub.knime.com/) to create an EC. A list of all available default KNIME docker images can be found [here](https://docs.knime.com/latest/business_hub_admin_guide/index.html#docker-executor-images).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to KNIME Document

With the release of 5.3.2 the docker image name does not need the registry.hub.knime.com prefix anymore. Find more information about KNIME Executor images provided by KNIME here.

After release of 5.3.1, all the KNIME docker images are hosted on Docker Hub instead.


However, the official Docker images hold KNIME Extensions only; no Community (trusted or experimental) or Partner Extensions included. Since the "Geospatial Analytics Extension for KNIME" is a Trusted Community Extension, it needs to be integrated into the Docker image, which requires to enhance the official image and build your own custom Docker image.

## Requirements
### Options
This can be achieved in two ways.
1. **Automatic**
- One can use the [_"Executor Image Builder"_](https://hub.knime.com/knime/spaces/KNIME%20Business%20Hub%20Admin%20Workflows/Executor%20Image%20Builder~V8b8qdFAdrBOziiD/current-state) Data App that is part of the Administrator Workflows for KNIME Business Hub can be found [here](https://hub.knime.com/knime/spaces/KNIME%20Business%20Hub%20Admin%20Workflows/~6vfL9nZbxwU3yNbU/).
- This currently only works for online installations. For airgap installations, please refer to the manual steps.
- See below, for more details.
2. **Manual**
- One can use the default KNIME docker images (mentioned above), and build on top of it. This requires to create a Dockerfile that pulls the default image and adds the extension while building the custom image.
- See below, for more details.

### Docker Image Registry
In either way, one needs to configure a docker V2 registry to be able to provide the custom image to the KNIME Businss Hub. Here, one can either use the [embedded registry](https://docs.knime.com/latest/business_hub_installation_guide/index.html#embedded-docker-registry-configuration) of the KNIME Business Hub itself, or use an [external registry](https://docs.docker.com/get-started/docker-concepts/the-basics/what-is-a-registry/) (like ECR, ACR, etc.). The latter link also explains what a docker image registry is in general.

In case, you are not using the embedded registry but an external one, you can configure your KNIME Business Hub to use that within the KOTS Admin Console.

![registry-settings](./imgs/registry-settings.png)

Please also refer to the official documentation on [Connecting to an External Registry](https://docs.replicated.com/vendor/packaging-private-images).

## Automatic Image Creation
This is the easiest and most automatic way to provide a docker image that includes the Geospatial Extension to the KNIME Business Hub. This will allow you to
- create the Dockerfile,
- build an executor docker image,
- push it to a registry, and eventually
- make it available when creating or editing an execution context.

Our [documentation](https://docs.knime.com/latest/business_hub_admin_guide/index.html#build-dockerfile-with-data-app) describes the application step by step.

### Requirements
Our [documentation](https://docs.knime.com/latest/business_hub_admin_guide/index.html#add-extension-docker-executor-images) describes the necessary requirements. Among them, one needs
- to enable the Execution Image Builder within the ```KOTS Admin Console``` > ```Config``` > ```Execution Image Builder``` > check ```Enable Execution Image Builder```
- global admin priviliges
- global admin application password

## Manual Image Creation
If it is desired or necessary to not to go with the Data App (e.g. due to an airgap installation or technical preference), one can do the following steps manually:
- create the Dockerfile,
- build an executor docker image,
- push it to a registry, and eventually
- make it available when creating or editing an execution context

Besides the below steps, our [documentation](https://docs.knime.com/latest/business_hub_admin_guide/index.html#build-image-manually) also describes steps around manual docker image creation.

### Requirements
To be able to build your custom Docker image it is required to run [Docker](https://www.docker.com/), more precisely the [Docker Engine](https://docs.docker.com/engine/install/).
Please make yourself familiar with the install instructions according to your operating system.
In general, you can build the image locally. It is not required that the image is built somewhere on the KNIME Business Hub.

## Dockerfile
### Dockerfile
The basis of a Docker image is a Dockerfile. It defines
- the source image (in this case the official Docker image from KNIME; check out all available images [here](https://docs.knime.com/latest/business_hub_admin_guide/index.html#docker-executor-images)),
- the source image (in this case the offical Docker image from KNIME; check out all available images [here](https://docs.knime.com/latest/business_hub_admin_guide/index.html#docker-executor-images)),
- the libraries/packages to be installed, and subsequently
- which extensions should be installed into the executor.

Make sure to save that file with the name ```Dockerfile``` and add no file extension.

Within the Dockerfile, we define two variables at the bottom that are important while installing the extension(s):

```KNIME_UPDATE_SITES``` - the URL to the necessary Update Site(s)
```KNIME_UPDATE_SITES``` the URL to the necessary Update Site(s)

```KNIME_FEATURES``` - the Feature ID of the extension to be installed
```KNIME_FEATURES``` the Feature ID of the extension to be installed

Check out [this video](https://www.youtube.com/watch?v=-dO79Id3VAo&t=143s) from the KNIMETV channel that helps you to find the right update site URL and the Feature ID.
Check out [this video](https://www.youtube.com/watch?v=-dO79Id3VAo&t=143s) from the KNIMETV channel that helps you to find the right update site URL and the feature ID.

# Define the base image
FROM registry.hub.knime.com/knime/knime-full:r-5.2.1-369
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After release of 5.3.1, you can use from Docker hub directly and skip the registry.

With the release of 5.3.2 the docker image name does not need the registry.hub.knime.com prefix anymore. Find more information about KNIME Executor images provided by KNIME here.
use something like,
knime/knime-full:r-5.3.3-666

Expand Down Expand Up @@ -52,30 +96,22 @@ Check out [this video](https://www.youtube.com/watch?v=-dO79Id3VAo&t=143s) from
# Execute extension installation script
RUN ./install-extensions.sh

## Build and push the Docker image
Now, the Docker image can be built from the Dockerfile and made available to the KNIME Business Hub with the following commands. Please make sure to adapt the following lines accordingly, when running them in the CLI of your laptop.
### Build and push the Docker image
Now, the Docker image can be build from the Dockerfile and made available to the KNIME Business Hub with the following commands.

# 1) Build the Docker image
# Build the Docker image
docker build . -f .\Dockerfile -t knime-full-geo:5.2.1 --no-cache

# 2) Retag the image to make it readable for KNIME Business Hub
docker tag knime-full-geo:5.2.1 registry.<base-url>/knime-full-geo:5.2.1

# 3) Login to the embedded registry of the KNIME Business Hub (if configured)
# Authenticate with the registry:
# You will need to provide a password. In case you do not have one you can manage this from the KOTS Admin Console > Config > Embedded Registry.
docker login --username <username> registry.<base-url>
# Retag the image to make it readable for KNIME Business Hub
docker tag knime-full-geo:5.2.1 registry.<hub-url>/knime-full-geo:5.2.1

# 4) Push the image to registry
docker push registry.<base-url>/knime-full-geo:5.2.1
# Login to the embedded registry of the KNIME Business Hub (if configured)
docker login --username <username> registry.<hub-url>

In case, you are not using the embedded registry but an external one, you can configure your KNIME Business Hub to use that within the KOTS Admin Console. Using an external docker registry, requires adapting the steps above accordingly.

![registry-settings](./imgs/registry-settings.png)

Please also refer to the official documentation on [Connecting to an External Registry](https://docs.replicated.com/vendor/packaging-private-images).
# Push the image to registry
docker push registry.<hub-url>/knime-full-geo:5.2.1

## Check if the Docker image is available on the registry
### Check if the Docker image is available on the registry
Once the upload is finished you can double check if that image can be found on the registry by searching the Docker registry API. Therefore, it is basically enough to use your web browser.

# Check which image names are available
Expand All @@ -94,4 +130,4 @@ Once the upload is finished you can double check if that image can be found on t
## Creating an Execution Context
Now you can apply the Docker image and create an EC from it by passing the registry string under the "Docker image" section while creating an EC. Please read the [KNIME docs](https://docs.knime.com/latest/business_hub_user_guide/index.html#execution_contexts) to find out more about how to setup a new execution context.

![create-execution-context](./imgs/create-execution-context.png)
![create-execution-context](./imgs/create-execution-context.png)