feat: add profile to launch harbor as external registry (#416)

* feat: update `k3-create.sh`

Signed-off-by: Guilhem Barthés <[email protected]>

* doc: add page about how to use harbor

Signed-off-by: Guilhem Barthés <[email protected]>

* fix: wrong hot modification

Signed-off-by: Guilhem Barthés <[email protected]>

* fix: add placeholder for `HARBOR_REGISTRY_DOMAIN`

Signed-off-by: Guilhem Barthés <[email protected]>

* fix: format

Signed-off-by: Guilhem Barthés <[email protected]>

* chore: add precision to harbor deployment doc

Signed-off-by: SdgJlbl <[email protected]>

---------

Signed-off-by: Guilhem Barthés <[email protected]>
Signed-off-by: SdgJlbl <[email protected]>
Co-authored-by: SdgJlbl <[email protected]>

Author: guilhem-barthes

docs/source/how-to/developing-substra/harbor.rst

@@ -0,0 +1,129 @@
+**************************
+Harbor in local deployment
+**************************
+
+
+
+This page describes how to use Harbor in our development setup, using skaffold. It is divided in two categories: the first one describes how to use a harbor registry inside the `k3d` cluster, meanwhile the second describes how to use a harbor registry outside of the cluster. The third section describes the set-up that will be needed.
+
+
+
+Use in-cluster Harbor
+=====================
+
+First-time set-up (specific to in-cluster)
+------------------------------------------
+
+As ``sudo``, add ``127.0.0.1       registry.org-2.com`` to ``/etc/hosts``
+
+      .. code:: bash
+
+         echo "127.0.0.1       registry.org-2.com" | sudo tee -a /etc/hosts
+
+Use
+---
+
+1. Set ``HARBOR_CERT_PATH`` to point to the absolute path to ``orchestrator/examples/tools/ca.crt``
+
+   .. code-block:: bash
+
+        export HARBOR_CERT_PATH=<absolute_path_to_orchestrator_repo>/orchestrator/examples/tools/ca.crt
+
+2. Re-create the cluster and launch skaffold on the orchestrator
+
+   .. code-block:: bash
+
+        ./k3-create.sh
+        cd orchestrator
+        skaffold run
+
+3. Start ``substra-backend`` with profile ``org-2-harbor``
+
+4. Activate port-forward (port ``30046``) on
+   ``harbor-nginx-xxxxxxxxxx-xxxxx`` (referred as ``<pod_name>``),
+   depending of your tool:
+
+   -  ``kubectl``:
+
+      .. code:: bash
+
+         kubectl port-forward -n harbor deployments/harbor-nginx 30046:https
+
+   -  ``k9s``:
+
+      1. Hover pod ``<pod_name>``
+      2. Press ``<SHIFT>+F``
+      3. Replace ``nginx::30000`` by ``nginx::30046`` (the
+         ``Local port`` should be replaced by ``30046``)
+
+5. Follow the instructions in Harbor-set-up with the following informations:
+
+   -  URL: https://registry.org-2.com:30046
+   -  Identifier: ``admin``
+   -  Password: ``harborP@ssword2403``
+
+Use external Harbor
+===================
+
+1. Follow the Harbor-set-up
+
+2. Create ``docker-config`` secret
+
+ .. code:: bash
+      
+      kubectl create secret docker-registry docker-config -n org-2 --docker-server=<your-registry-server> --docker-username=<your-name>
+
+3. Update your value file (you can use ``backend-org-2-harbor.yaml`` as a model)
+
+   .. code:: yaml
+
+      docker-registry:
+        enabled: false
+
+      containerRegistry:
+        local: false
+        scheme: 
+        host: # The host, without the port, as it is defined in the field port
+        pullDomain: # The harbor host, with the port as it won't use the port field
+        port: 443
+
+      kaniko:
+        dockerConfigSecretName: docker-config # Equals to the name given to the secret at the previous step
+
+
+Harbor-set-up
+=============
+
+1. In the frontend, create project ``substra``
+
+2. Set the variable ```HARBOR_REGISTRY_DOMAIN``` to your registry URL
+
+    .. code-block:: bash
+
+        export HARBOR_REGISTRY_DOMAIN=<registry_url>
+
+3. Login to the registry
+
+    .. code-block:: bash
+
+        docker login $HARBOR_REGISTRY_DOMAIN
+
+4. Manually add base image
+
+   1. Pull image
+
+      .. code:: bash
+
+         docker pull ghcr.io/substra/substra-tools:latest
+
+   2. Tag it
+
+      .. code:: bash
+
+         docker tag ghcr.io/substra/substra-tools:latest $HARBOR_REGISTRY_DOMAIN/substra/substra-tools:latest
+
+   3. Push to repository
+
+      .. code:: bash
+
+         docker push $HARBOR_REGISTRY_DOMAIN/substra/substra-tools:latest

docs/source/how-to/developing-substra/index.rst

@@ -6,4 +6,5 @@ The following guides might be of interest if you are interested in contributing
 .. toctree::
    :maxdepth: 1
 
-   local-deployment.rst
+   local-deployment.rst
+   harbor.rst

docs/source/how-to/developing-substra/local-deployment/k3-create.sh

@@ -11,7 +11,13 @@ k3d cluster delete || echo 'No cluster'
 mkdir -p /tmp/org-1
 mkdir -p /tmp/org-2
 mkdir -p /tmp/org-3
-k3d cluster create --api-port 127.0.0.1:6443 -p 80:80@loadbalancer -p 443:443@loadbalancer --k3s-arg "--disable=traefik,metrics-server@server:*" --volume /tmp/org-1:/tmp/org-1 --volume /tmp/org-2:/tmp/org-2 --volume /tmp/org-3:/tmp/org-3
+args=()
+
+if [ "$HARBOR_CERT_PATH" ]; then
+   args+=("--volume" "${HARBOR_CERT_PATH}:/etc/ssl/certs/harbor.crt")
+fi
+
+k3d cluster create --api-port 127.0.0.1:6443 -p 80:80@loadbalancer -p 443:443@loadbalancer --k3s-arg "--disable=traefik,metrics-server@server:*" --volume /tmp/org-1:/tmp/org-1 --volume /tmp/org-2:/tmp/org-2 --volume /tmp/org-3:/tmp/org-3 "${args[@]}"
 
 # Patch and install nginx-ingress
 curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/kind/deploy.yaml > /tmp/deploy.yaml