-
Notifications
You must be signed in to change notification settings - Fork 11
Add info about Executor Builder App to the installation guide #371
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,29 +1,73 @@ | ||
# Setup Custom Executor Docker Image with the Geospatial Extension for KNIME Business Hub | ||
|
||
Running a workflow on KNIME Business Hub requires an Execution Context (EC). An EC runs one or multiple pods. A pod holds Docker containers in turn, which run Docker images in the end. In general, one can simply use the official Docker images from the public [KNIME Artifact Registry](https://registry.hub.knime.com/) to create an EC. | ||
## Introduction | ||
Running a workflow on KNIME Business Hub requires an Execution Context (EC). Inside that EC runs one or multiple pods - that hold Docker containers - which run Docker images. In general, one can simply use the official Docker images from the public [KNIME Artifact Registry](https://registry.hub.knime.com/) to create an EC. A list of all available default KNIME docker images can be found [here](https://docs.knime.com/latest/business_hub_admin_guide/index.html#docker-executor-images). | ||
|
||
However, the official Docker images hold KNIME Extensions only; no Community (trusted or experimental) or Partner Extensions included. Since the "Geospatial Analytics Extension for KNIME" is a Trusted Community Extension, it needs to be integrated into the Docker image, which requires to enhance the official image and build your own custom Docker image. | ||
|
||
## Requirements | ||
### Options | ||
This can be achieved in two ways. | ||
1. **Automatic** | ||
- One can use the [_"Executor Image Builder"_](https://hub.knime.com/knime/spaces/KNIME%20Business%20Hub%20Admin%20Workflows/Executor%20Image%20Builder~V8b8qdFAdrBOziiD/current-state) Data App that is part of the Administrator Workflows for KNIME Business Hub can be found [here](https://hub.knime.com/knime/spaces/KNIME%20Business%20Hub%20Admin%20Workflows/~6vfL9nZbxwU3yNbU/). | ||
- This currently only works for online installations. For airgap installations, please refer to the manual steps. | ||
- See below, for more details. | ||
2. **Manual** | ||
- One can use the default KNIME docker images (mentioned above), and build on top of it. This requires to create a Dockerfile that pulls the default image and adds the extension while building the custom image. | ||
- See below, for more details. | ||
|
||
### Docker Image Registry | ||
In either way, one needs to configure a docker V2 registry to be able to provide the custom image to the KNIME Businss Hub. Here, one can either use the [embedded registry](https://docs.knime.com/latest/business_hub_installation_guide/index.html#embedded-docker-registry-configuration) of the KNIME Business Hub itself, or use an [external registry](https://docs.docker.com/get-started/docker-concepts/the-basics/what-is-a-registry/) (like ECR, ACR, etc.). The latter link also explains what a docker image registry is in general. | ||
|
||
In case, you are not using the embedded registry but an external one, you can configure your KNIME Business Hub to use that within the KOTS Admin Console. | ||
|
||
 | ||
|
||
Please also refer to the official documentation on [Connecting to an External Registry](https://docs.replicated.com/vendor/packaging-private-images). | ||
|
||
## Automatic Image Creation | ||
This is the easiest and most automatic way to provide a docker image that includes the Geospatial Extension to the KNIME Business Hub. This will allow you to | ||
- create the Dockerfile, | ||
- build an executor docker image, | ||
- push it to a registry, and eventually | ||
- make it available when creating or editing an execution context. | ||
|
||
Our [documentation](https://docs.knime.com/latest/business_hub_admin_guide/index.html#build-dockerfile-with-data-app) describes the application step by step. | ||
|
||
### Requirements | ||
Our [documentation](https://docs.knime.com/latest/business_hub_admin_guide/index.html#add-extension-docker-executor-images) describes the necessary requirements. Among them, one needs | ||
- to enable the Execution Image Builder within the ```KOTS Admin Console``` > ```Config``` > ```Execution Image Builder``` > check ```Enable Execution Image Builder``` | ||
- global admin priviliges | ||
- global admin application password | ||
|
||
## Manual Image Creation | ||
If it is desired or necessary to not to go with the Data App (e.g. due to an airgap installation or technical preference), one can do the following steps manually: | ||
- create the Dockerfile, | ||
- build an executor docker image, | ||
- push it to a registry, and eventually | ||
- make it available when creating or editing an execution context | ||
|
||
Besides the below steps, our [documentation](https://docs.knime.com/latest/business_hub_admin_guide/index.html#build-image-manually) also describes steps around manual docker image creation. | ||
|
||
### Requirements | ||
To be able to build your custom Docker image it is required to run [Docker](https://www.docker.com/), more precisely the [Docker Engine](https://docs.docker.com/engine/install/). | ||
Please make yourself familiar with the install instructions according to your operating system. | ||
In general, you can build the image locally. It is not required that the image is built somewhere on the KNIME Business Hub. | ||
|
||
## Dockerfile | ||
### Dockerfile | ||
The basis of a Docker image is a Dockerfile. It defines | ||
- the source image (in this case the official Docker image from KNIME; check out all available images [here](https://docs.knime.com/latest/business_hub_admin_guide/index.html#docker-executor-images)), | ||
- the source image (in this case the offical Docker image from KNIME; check out all available images [here](https://docs.knime.com/latest/business_hub_admin_guide/index.html#docker-executor-images)), | ||
- the libraries/packages to be installed, and subsequently | ||
- which extensions should be installed into the executor. | ||
|
||
Make sure to save that file with the name ```Dockerfile``` and add no file extension. | ||
|
||
Within the Dockerfile, we define two variables at the bottom that are important while installing the extension(s): | ||
|
||
```KNIME_UPDATE_SITES``` - the URL to the necessary Update Site(s) | ||
```KNIME_UPDATE_SITES``` the URL to the necessary Update Site(s) | ||
|
||
```KNIME_FEATURES``` - the Feature ID of the extension to be installed | ||
```KNIME_FEATURES``` the Feature ID of the extension to be installed | ||
|
||
Check out [this video](https://www.youtube.com/watch?v=-dO79Id3VAo&t=143s) from the KNIMETV channel that helps you to find the right update site URL and the Feature ID. | ||
Check out [this video](https://www.youtube.com/watch?v=-dO79Id3VAo&t=143s) from the KNIMETV channel that helps you to find the right update site URL and the feature ID. | ||
|
||
# Define the base image | ||
FROM registry.hub.knime.com/knime/knime-full:r-5.2.1-369 | ||
|
||
|
@@ -52,30 +96,22 @@ Check out [this video](https://www.youtube.com/watch?v=-dO79Id3VAo&t=143s) from | |
# Execute extension installation script | ||
RUN ./install-extensions.sh | ||
|
||
## Build and push the Docker image | ||
Now, the Docker image can be built from the Dockerfile and made available to the KNIME Business Hub with the following commands. Please make sure to adapt the following lines accordingly, when running them in the CLI of your laptop. | ||
### Build and push the Docker image | ||
Now, the Docker image can be build from the Dockerfile and made available to the KNIME Business Hub with the following commands. | ||
|
||
# 1) Build the Docker image | ||
# Build the Docker image | ||
docker build . -f .\Dockerfile -t knime-full-geo:5.2.1 --no-cache | ||
|
||
# 2) Retag the image to make it readable for KNIME Business Hub | ||
docker tag knime-full-geo:5.2.1 registry.<base-url>/knime-full-geo:5.2.1 | ||
|
||
# 3) Login to the embedded registry of the KNIME Business Hub (if configured) | ||
# Authenticate with the registry: | ||
# You will need to provide a password. In case you do not have one you can manage this from the KOTS Admin Console > Config > Embedded Registry. | ||
docker login --username <username> registry.<base-url> | ||
# Retag the image to make it readable for KNIME Business Hub | ||
docker tag knime-full-geo:5.2.1 registry.<hub-url>/knime-full-geo:5.2.1 | ||
|
||
# 4) Push the image to registry | ||
docker push registry.<base-url>/knime-full-geo:5.2.1 | ||
# Login to the embedded registry of the KNIME Business Hub (if configured) | ||
docker login --username <username> registry.<hub-url> | ||
|
||
In case, you are not using the embedded registry but an external one, you can configure your KNIME Business Hub to use that within the KOTS Admin Console. Using an external docker registry, requires adapting the steps above accordingly. | ||
|
||
 | ||
|
||
Please also refer to the official documentation on [Connecting to an External Registry](https://docs.replicated.com/vendor/packaging-private-images). | ||
# Push the image to registry | ||
docker push registry.<hub-url>/knime-full-geo:5.2.1 | ||
|
||
## Check if the Docker image is available on the registry | ||
### Check if the Docker image is available on the registry | ||
Once the upload is finished you can double check if that image can be found on the registry by searching the Docker registry API. Therefore, it is basically enough to use your web browser. | ||
|
||
# Check which image names are available | ||
|
@@ -94,4 +130,4 @@ Once the upload is finished you can double check if that image can be found on t | |
## Creating an Execution Context | ||
Now you can apply the Docker image and create an EC from it by passing the registry string under the "Docker image" section while creating an EC. Please read the [KNIME docs](https://docs.knime.com/latest/business_hub_user_guide/index.html#execution_contexts) to find out more about how to setup a new execution context. | ||
|
||
 | ||
 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
According to KNIME Document
After release of 5.3.1, all the KNIME docker images are hosted on Docker Hub instead.