Arkade Install Kafka fails

[remiconnesson]

Expected Behaviour

arkade install kafka

Current Behaviour

VALUES values.yaml

Command: /home/remi/.arkade/bin/helm [upgrade --install cp-helm-charts confluentinc/cp-helm-charts --namespace  --values /tmp/charts/cp-helm-charts/values.yaml --set cp-zookeeper.servers=1 --set cp-kafka.enabled=true --set cp-schema-registry.enabled=false --set cp-kafka-rest.enabled=false --set cp-kafka-connect.enabled=false --set cp-ksql-server.enabled=false --set cp-control-center.enabled=false --set cp-zookeeper.enabled=true --set cp-kafka.brokers=1]

Release "cp-helm-charts" does not exist. Installing it now.

Error: unable to build kubernetes objects from release manifest: resource mapping not found for name: "cp-helm-charts-cp-zookeeper-pdb" namespace: "" from "": no matches for kind "PodDisruptionBudget" in version "policy/v1beta1"
ensure CRDs are installed first

Error: exit code 1, stderr: Error: unable to build kubernetes objects from release manifest: resource mapping not found for name: "cp-helm-charts-cp-zookeeper-pdb" namespace: "" from "": no matches for kind "PodDisruptionBudget" in version "policy/v1beta1"
ensure CRDs are installed first

Are you a GitHub Sponsor yet (Yes/No?)

• Yes
• No

Possible Solution

Steps to Reproduce (for bugs)

1. arkade get k3sup
2. export PATH=$PATH:$HOME/.arkade/bin/ in .zshrc, then source
3. k3sup install --local --k3s-extra-args "--write-kubeconfig-mode=644"
4. (wait)
5. arkade install kafka
6. (above results in an error)

Context

I'm using arkade because it's the only tool I understand yet, I'm not versed in helm yet.
So basically my only shot to install kafka on k3s

This being said, not really affecting me I'm using it for a toy project

I'm opening this issue fyi because every install I did with arkade worked out of the box, it's the first time it doesn't. Beautiful tool.

Your Environment

• What Kubernetes distribution are you using?

K3S on ubuntu 22 single node

WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.5+k3s2", GitCommit:"de654222cb2c0c21776c32c26505fb684b246a1b", GitTreeState:"clean", BuildDate:"2023-01-11T21:23:33Z", GoVersion:"go1.19.4", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.5+k3s2", GitCommit:"de654222cb2c0c21776c32c26505fb684b246a1b", GitTreeState:"clean", BuildDate:"2023-01-11T21:23:33Z", GoVersion:"go1.19.4", Compiler:"gc", Platform:"linux/amd64"}

• Operating System and version (e.g. Linux, Windows, MacOS):

➜  ~ uname -a
Linux <redacted> 5.15.0-46-generic #49-Ubuntu SMP Thu Aug 4 18:03:25 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
➜  ~ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.1 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

• What arkade version is this?

Version: 0.8.59
Git Commit: bdb3869d5cf87ccc0d1d2ec0c9ecffe113e31ce6```

[alexellis]

Thanks for the feedback.

@Jasstkn is this something you could try out and look into?

[Jasstkn]

Thanks for the feedback.

@Jasstkn is this something you could try out and look into?

I'll take a look later today.

[Jasstkn]

@remiconnesson @alexellis the reason is the following: kafka's charts aren't well maintained. There is a lot of issues that are currently opened. One of them addresses issue with PDB that you faced: confluentinc/cp-helm-charts#629.

It means that arkade install kafka won't work on the Kubernetes v1.21+

[remiconnesson]

Thank you very much for answering that fast

In the meantime, For anyone reading this wanting to install kafka on k3s I'm currently looking into this solution:

• https://github.com/IBM/kraft-mode-kafka-on-kubernetes (to create the docker image)
• https://learnk8s.io/kafka-ha-kubernetes (to show how to use it)

[alexellis]

@remiconnesson Confluent provide free Kafka for development. Have you looked into that? We use it for testing the OpenFaaS Pro Kafka Connector

[alexellis]

Any other suggestions for charts @Jasstkn ?

[Jasstkn]

Any other suggestions for charts @Jasstkn ?

https://github.com/bitnami/charts/tree/main/bitnami/kafka?

[mrwormhole]

looks like it is using "policy/v1beta1", it is deprecated in v1.21 and completely removed in v1.25 in favor of "policy/v1", mostly helm chart should use this in PodSecurityPolicy or PodDisruptionBudget

i would use bitnami charts instead of confluentinc ones, they are more maintained @Jasstkn

[rgee0]

From https://github.com/confluentinc/cp-helm-charts:

Confluent Platform Helm Charts [Deprecated]
Deprecated: The cp-helm-charts is deprecated in favor of Confluent For Kubernetes .

[alexellis]

What about the one Bitnami maintains? https://github.com/bitnami/charts/tree/main/bitnami/kafka

This is more similar in approach than moving to using an operator to install/manage Kafka.

[joebowbeer]

@alexellis wrote:

What about the one Bitnami maintains? https://github.com/bitnami/charts/tree/main/bitnami/kafka

This is more similar in approach than moving to using an operator to install/manage Kafka.

Note that bitnami no longer provides a free kafka image:

bitnami/containers#83267

I think Strimzi is the best way to install Apache Kafka on k8s:

https://strimzi.io/docs/operators/latest/deploying#con-strimzi-installation-methods_str