Add Talos support

[johanngyger]

Talos doesn't have a shell, so it's currently impossible to install the driver. Two PRs help address the issue; one was made years ago:

• Working prototype for Talos with support for NFS, iSCSI, and NVMe/TCP #163
• Talos fix #47

Please let me know if I can help. I won't have a test system at hand in the future, though (PR 163 was tested with a Dorado 5000 V6).

[nyanmark]

I wanted to try your patch however, I can't really get the container to run properly I just get huawei-csi: no such file or directory, even though the file is definitely there and executable, after adding bash to the alpine container and executing in that I get cannot execute: required file not found. Anyways, it would be amazing if this CSI supported talos.

[johanngyger]

@nyanmark I've pushed the manifests that I've used for testing; please check them out.

• manual/esdk/deploy/huawei-csi-controller.yaml
• manual/esdk/deploy/huawei-csi-node.yaml

(I used release 4.7 but I patched the image for the huawei-csi-driver which is available as a container image here: registry.gitlab.com/johanngyger/talos-huawei-csi:42)

You also have to make sure that the namespace allows priviledged containers: kubectl label namespace huawei-csi pod-security.kubernetes.io/enforce=privileged

After that, could you please provide more info on what is failing like this?

• kubectl logs huawei-csi-controller-... huawei-csi-driver
• kubectl logs huawei-csi-node-... huawei-csi-driver

[nyanmark]

The manifests you sent me with the image work, however, when using the build.sh script to create the archive of all the images the container fails with "exec /huawei-csi: no such file or directory", thats all I see in kubectl logs, the only thing I changed was added this line just like the original repo has so the build.sh script works properly

-FROM alpine
+FROM alpine:latest as huawei-csi-driver

The huawei-csi file does get put in the container, so it kinda makes little sense as to why it says no such file or directory

[nyanmark]

Ok, I kinda know why my locally built container doesn't work, its not really the patch but go, alpine and the way its built, as the build scripts build the binary locally, and then copy it into docker it gets upset over the musl environment.

ldd huawei-csi
statically linked

The binary built on my PC is clearly statically linked and executes, then this is the binary inside the alpine container

ldd huawei-csi
/lib64/ld-linux-x86-64.so.2

Ofc, the alpine container doesn't have an ld-linux so it fails to execute, adding gcompat to the container makes it execute just fine.

[johanngyger]

I used my own build script to only build the csi binary locally:

csi_build.sh:

#!/bin/bash

cd csi
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build
cd ..
docker build -t registry.gitlab.com/johanngyger/talos-huawei-csi:42 --platform linux/amd64 -f csi.Dockerfile .
docker push registry.gitlab.com/johanngyger/talos-huawei-csi:42

csi.Dockerfile:

FROM alpine
RUN apk update && apk add --no-cache xfsprogs xfsprogs-extra findmnt blkid
COPY csi/csi huawei-csi
ENTRYPOINT ["/huawei-csi"]

[nyanmark]

Well, this is what I got now

ARG VERSION

FROM alpine:latest as huawei-csi-driver

LABEL version="${VERSION}"
LABEL maintainers="Huawei eSDK CSI development team"
LABEL description="Kubernetes CSI Driver for Huawei Storage: $VERSION"

RUN apk update && apk add --no-cache xfsprogs xfsprogs-extra findmnt blkid gcompat grep
ARG binary=./huawei-csi
COPY ${binary} huawei-csi
ENTRYPOINT ["/huawei-csi"]

The reason the build doesn't work for me without gcompat is probably due to package versions rather than, the build script itself as the original makefile also uses CGO_ENABLED GOOS and GOARCH just like your build script. I'm guessing the alpine container just doesn't like the gobinary I build on my gentoo machine, well its only a guess because I haven't really used go nor do I understand it.

I'm trying to get it to work with fiber channel, also got it working with helm now, only issue is this

Run shell cmd "mkfs -t ext4 -F /dev/sdf" output: [/bin/sh: mkfs: not found], error: [exit status 127]

Setting the fstype to xfs in the storage class fixed it.

[johanngyger]

Ok, good to know. You'd have to install the Alpine packackge for other filesystems and adapt the Go code, I just got it working with xfs.

I also hat to tweak a bit and somtimes I executed commands directly in the shell so that it worked eventually:

/usr/bin/nsenter -i/proc/1/ns/ipc -n/proc/1/ns/net -u/proc/1/ns/uts -m/proc/1/ns/mnt /usr/local/sbin/nvme list-subsys -o json
/usr/bin/nsenter -i/proc/1/ns/ipc -n/proc/1/ns/net -u/proc/1/ns/uts -m/proc/1/ns/mnt /usr/local/sbin/nvme discover -t tcp -a ... -s 4420
/usr/bin/nsenter -i/proc/1/ns/ipc -n/proc/1/ns/net -u/proc/1/ns/uts -m/proc/1/ns/mnt /usr/local/sbin/nvme connect --transport=tcp --traddr=... --trsvcid=4420 --nqn=...

(These commands use the Talos extension for nvme-cli. Installing nvme in the container somehow didn't work because the mount namespace of proc 1 appears to be needed)