diff --git a/docs/architecture/identity.md b/docs/architecture/identity.md new file mode 100644 index 00000000..eb525a9e --- /dev/null +++ b/docs/architecture/identity.md @@ -0,0 +1,292 @@ +# Identity in Azimuth + +This document explains how identity is designed to work in Azimuth. + +## Personas + +Azimuth has two distinct personas that consume Azimuth and the platforms that are deployed +using it - **platform admins** and **platform users**. + +**Platform admins** are able to access the Azimuth portal where they can administer platforms +in their tenancies and the platform users that are able to access the services for those +platforms (e.g. Jupyter Notebooks, Guacamole web desktops, monitoring dashboards). + +**Platform users** are able to access platform services to which they have been granted access +by a platform admin. They do not need to be able to sign in to the Azimuth portal or even be +aware that the platform is deployed in an Azimuth tenancy. + +By default, platform admins are also platform users that are authorised to access all of the +services for platforms deployed in their tenancies. + +## Technologies + +Azimuth utilises Kubernetes Ingress, Keycloak and Zenith to implement secure access to Azimuth +and the platforms deployed using it. + +### Kubernetes Ingress + +Azimuth and Zenith leverage Kubernetes Ingress as a dynamically configurable HTTP(S) proxy to +route external traffic to the correct Kubernetes service based on the hostname and/or path. + +### Keycloak + +As part of an Azimuth installation, a Keycloak instance is deployed. Azimuth uses this Keycloak +instance to provide authentication and authorisation for platform services exposed using Zenith, +and optionally for Azimuth itself. + +### Zenith + +Zenith is a tunnelling HTTP(S) proxy that is used by Azimuth to expose platform services. Using +Zenith allows Azimuth to expose platform services that are behind a NAT or firewall without +consuming public IPs. + +Zenith uses a server/client model where a client initiates a secure SSH tunnel with the server +for the service that wants to be proxied. The server then forwards traffic for the proxied service +back down the tunnel. This is accomplished by creating a Kubernetes `Service` for each Zenith +service and using Kubernetes Ingress to route traffic to the correct service based on the hostname. + +Zenith is able to perform TLS termination and single sign-on using OpenID Connect (OIDC) for +proxied services before forwarding traffic to them. This is implemented by configuring the +Kubernetes `Ingress` resources appropriately - in particular the OIDC authentication and +authorisation is implemented using an auth subrequest. + +## Accessing Azimuth + +An Azimuth operator can configure authentication and authorization for Azimuth itself using +either OpenStack credentials directly or using a realm in the Azimuth-managed Keycloak. Each +mechanism has pros and cons. + +An "Azimuth tenancy" is essentially a `Namespace` in the Azimuth Kubernetes cluster that holds +the platform definitions and related resources for that tenancy (implemented as Kubernetes custom +resources). Currently, these tenancies have a one-to-one mapping with OpenStack projects. + +In order to manage platform resources in the underlying cloud on behalf of users, Azimuth needs +to be able to translate an authenticated Azimuth session into two things: + + 1. A list of Azimuth tenancies that a user is permitted to access + 2. A credential that is able to manage resources in the corresponding OpenStack project + +This section documents how this is acheived for each authentication method. + +### Using OpenStack credentials + +Azimuth allows authenticating directly using credentials for the underlying OpenStack cloud, +with support for domain/username/password, application credentials and Keystone federated +identity providers. + +In each case, Azimuth obtains a token that can be used to interact with the OpenStack APIs on +behalf of the user. This token is _unscoped_, i.e. not associated with a specific OpenStack +project, but can be used to query the OpenStack projects that the user has access to - these +projects correspond to the Azimuth tenancies that the user can access. Once the user enters +a tenancy context in Azimuth, the unscoped token can be exchanged for a _project-scoped_ token +that allows Azimuth to manage resources in the OpenStack project. + +When authenticating using OpenStack credentials, Azimuth will create the corresponding tenant +namespaces automatically when required. + +The following sequence diagram shows a typical flow for creating a platform in an Azimuth +tenancy using OpenStack authentication: + +```mermaid +sequenceDiagram + actor U as User + participant A as Azimuth Portal + participant N as Azimuth Tenant
Namespace + participant O as OpenStack + + U ->> A: Create platform in tenancy X + opt Session cookie not present or unscoped token expired + A ->> U: Authentication required + U ->> A: Visit login page + A ->> U: List of authentication methods + alt Domain/username/password or application credential selected + U ->> A: Supply credentials + A ->> O: Request unscoped token using supplied credentials + O ->> A: Return unscoped token for user + A -->> U: Redirect to tenancy
Set cookie containing unscoped token + else Keystone federated IDP selected + U ->> A: Keystone federated IDP selected + A -->> U: Redirect to Keystone federation URL + U ->> O: Visit Keystone federation URL + O ->> O: Authenticate user with external IDP
May require additional redirects, e.g. for OIDC + O ->> U: Return unscoped token to user's browser + U ->> A: Complete authentication by POSTing unscoped token + A -->> U: Redirect to tenancy
Set cookie containing unscoped token + end + U ->> A: Re-submit create platform in tenancy X + end + A ->> O: Exchange unscoped token for scoped token for project X + O ->> A: Return scoped token for project X + opt Azimuth tenant namespace does not exist + A ->> N: Create tenant namespace + N ->> A: Namespace created successfully + end + A ->> O: Create application credential for platform + O ->> A: Return platform application credential data + A ->> N: Write application credential to secret + N ->> A: Secret written successfully + A ->> N: Create custom resources for platform + N ->> A: Resources created successfully + A ->> U: Platform created in tenancy X +``` + +### Using a Keycloak realm + +!!! warning "Technology preview" + + This authentication method is currently in technology preview and not recommended for + production. + + Fully automated configuration of the Keycloak realm is not yet supported in `azimuth-ops`, + so it is recommended that you enable backups for your Azimuth installation in order to + preserve the Keycloak database in the case of an unrecoverable failure. + +Azimuth also supports authenticating against a special realm in the Keycloak instance that is +deployed as part of an Azimuth installation. When using this authentication method, it is no +longer required that platform admins have accounts on the underlying OpenStack cloud - the +Azimuth operator can configure the Keycloak realm with any identity providers (e.g. Microsoft +Entra, Google, LDAP, OIDC) and policies (e.g. MFA) that are supported by Keycloak that they +need for users signing in to Azimuth. + +In this authentication method, an Azimuth tenant namespace must be _pre-created_ for each +OpenStack project that you want to use with Azimuth and a group must exist in the Keycloak +realm that is used to grant access to this tenancy - `azimuth-ops` handles the creation of +these namespaces and groups for the projects listed in your Azimuth configuration. Keycloak +can be configured to automatically map users from federated identity providers into these +groups if desired, or users can be manually added to the groups as needed. + +Azimuth authenticates users against the Keycloak realm using OpenID Connect, and the `groups` +claim from the resulting token is used to determine which Azimuth tenancies the user is +permitted to access. + +#### Obtaining an OpenStack credential + +Because users no longer have OpenStack credentials, Azimuth must now own the credentials that +are used to provision resources in the underlying cloud on behalf of users. For each OpenStack +project that you want to use Azimuth with, an application credential must be _pre-created_ and +stored in a `Secret` in the corresponding Azimuth tenant namespace. `azimuth-ops` handles the +creation of these secrets, but not the creation of the application credentials themselves - +these must be created ahead of time and placed in your Azimuth configuration. + +!!! warning "Unrestricted application credentials" + + The application credentials must be **unrestricted**, meaning that they have the ability + to create additional application credentials. This is so that Azimuth can still create the + per-platform application credentials that are used to manage platform resources. + +!!! tip "Service account recommended" + + It is recommended that Azimuth has a service account in the underlying OpenStack that owns + the application credentials used by Azimuth. This service account will need to belong to all + the projects that you wish to use Azimuth with. + + This avoids all of the application credentials used by Azimuth being tied to an individual + OpenStack user who may have access to the cloud revoked, e.g. if they leave the organisation. + +The following sequence diagram shows a typical flow for creating a platform in an Azimuth +tenancy using the Keycloak realm for authentication: + +```mermaid +sequenceDiagram + actor U as User + participant A as Azimuth Portal + participant N as Azimuth Tenant
Namespace + participant K as Keycloak Realm + participant O as OpenStack + + U ->> A: Create platform in tenancy X + opt Session cookie not present or ID token invalid or expired + A ->> U: Authentication required + U ->> A: Visit login page + A -->> U: Redirect to OIDC authorize URL + U ->> K: Visit OIDC authorize URL + opt User does not have a current Keycloak session + K -->> U: Redirect to login page + U ->> K: Visit login page + K ->> K: Authenticate user
This may redirect to an external IDP + end + K -->> U: Redirect to OIDC callback URL + U ->> A: Visit OIDC callback URL + A ->> K: Request OIDC ID token + K ->> A: Return OIDC ID token with groups claim + A -->> U: Redirect to tenancy
Set cookie containing ID token + U ->> A: Re-submit create platform in tenancy X + end + A ->> A: Verify that user is authorised for tenancy X
using groups claim from OIDC token + A ->> N: Read project application credential secret + N ->> A: Return project application credential data + A ->> O: Create application credential for platform + O ->> A: Return platform application credential data + A ->> N: Write platform application credential to secret + N ->> A: Secret written successfully + A ->> N: Create custom resources for platform + N ->> A: Resources created successfully + A ->> U: Platform created in tenancy X +``` + +## Accessing platform services + +Access to platform services in Azimuth is mediated using Keycloak, regardless of how the user +authenticates with Azimuth and the mechanism for obtaining a cloud credential. + +Azimuth manages a realm in Keycloak for each Azimuth tenancy, and the platform users for a +tenancy are users that exist in the associated Keycloak realm. These users can be local Keycloak +users or users from a federated identity provider such as Microsoft Entra, Google or LDAP. + +By default, this realm has a single federated IDP that allows users belonging to the tenancy to +sign in using their existing Azimuth session (i.e. platform admins). Users that sign in using +this method are automatically added to an `admins` group that allows them to administer the realm +and grants access to all platform services for the tenancy. Platform admins are able to manage +local users and federated identity providers in this realm as required for their needs. + +Within this realm, Azimuth then manages an OpenID Connect client for each platform service and +a group that is used to grant access to that service. Zenith is configured to use this OIDC +client and group when performing authentication and authorisation for the service. + +Platform admins can grant a platform user access to a platform service by adding them to the +group for that service. Keycloak has powerful features for automatically mapping users from +federated identity providers into groups if desired. + +The following diagram shows the sequence of actions for a user accessing a platform service: + +```mermaid +sequenceDiagram + actor U as User + participant P as Zenith Proxy + participant R as Keycloak Realm + participant A as Azimuth Portal + participant S as Platform Service + + U ->> P: Visit service URL + opt OIDC ID token cookie not present or ID token invalid or expired + P -->> U: Redirect to OIDC authorize URL + U ->> R: Visit OIDC authorize URL + opt User does not have a current Keycloak session + R -->> U: Redirect to login page + U ->> R: Visit login page + alt User selects Azimuth IDP + R -->> U: Redirect to Azimuth Portal + U ->> A: Authenticate with Azimuth Portal + A ->> A: Verify user belongs
to tenancy for realm + A -->> U: Redirect to Keycloak realm + U ->> R: Complete authentication + else User selects another authentication method + R ->> R: Authenticate user
This may redirect to an external IDP + end + end + R -->> U: Redirect to OIDC callback URL + U ->> P: Visit OIDC callback URL + P ->> R: Request OIDC ID token + R ->> P: Return OIDC ID token with groups claim + P -->> U: Redirect to service URL, setting OIDC ID token cookie + U ->> P: Visit service URL with valid OIDC ID token cookie + end + P ->> P: Check groups claim of ID token for required groups + alt User is authorised + P ->> S: Forward request to proxied service + S ->> P: Response from proxied service + P ->> U: Return response to user + else User is not authorised + P ->> U: Return error page + end +``` diff --git a/docs/architecture/index.md b/docs/architecture/index.md new file mode 100644 index 00000000..218ae032 --- /dev/null +++ b/docs/architecture/index.md @@ -0,0 +1,3 @@ +# Azimuth Architecture + +Coming soon! diff --git a/docs/best-practice.md b/docs/best-practice.md index 167dd6e8..beb46109 100644 --- a/docs/best-practice.md +++ b/docs/best-practice.md @@ -102,7 +102,7 @@ environments. The `ansible-playbook` command should **never** be executed manual The recommended approach is to automatically deploy an independent `aio` environment for each feature branch, also known as -[per-branch dynamic review environments](deployment/automation/#per-branch-dynamic-review-environments). +[per-branch dynamic review environments](deployment/automation.md#per-branch-dynamic-review-environments). This allows changes to be validated before they are merged to `main`. Once a change is merged to `main`, it will be deployed automatically to the `staging` environment. diff --git a/docs/configuration/03-kubernetes-config.md b/docs/configuration/03-kubernetes-config.md index e8cca08d..73e460ef 100644 --- a/docs/configuration/03-kubernetes-config.md +++ b/docs/configuration/03-kubernetes-config.md @@ -95,7 +95,7 @@ to use volume-backed instances instead. !!! tip "etcd on a separate block device" If you only have a limited amount of SSD or, even better, local disk, available, - consider placing [etcd on a separate block device](#etcd-block-device) to make + consider placing [etcd on a separate block device](#etcd-configuration) to make best use of the limited capacity. To configure Kubernetes clusters to use volume-backed instances (i.e. use a Cinder diff --git a/docs/deployment/automation.md b/docs/deployment/automation.md index 88d41206..c8812a01 100644 --- a/docs/deployment/automation.md +++ b/docs/deployment/automation.md @@ -16,7 +16,7 @@ environments, although deployments to production typically include a manual appr !!! tip "Using a site mixin" To get the maximum benefit from automated deployments and the - [feature branch workflow](../repository/index.md#making-changes-to-your-environment), + [feature branch workflow](../repository/index.md#making-changes-to-your-configuration), you should try to minimise the differences between the production, staging and [dynamic review](#per-branch-dynamic-review-environments) environments. diff --git a/docs/deployment/testing.md b/docs/deployment/testing.md index 4082a3a3..7a214314 100644 --- a/docs/deployment/testing.md +++ b/docs/deployment/testing.md @@ -149,7 +149,7 @@ clouds: ## Generating and executing tests Before tests can be generated and executed for an [environment](../environments.md), the -environment must be successfully deployed either [manually](../) or by +environment must be successfully deployed either [manually](./index.md) or by [automation](./automation.md). You must also have the Python dependencies installed on the machine that the tests will be diff --git a/docs/developing/index.md b/docs/developing/index.md index 2a6de917..36ba7acc 100644 --- a/docs/developing/index.md +++ b/docs/developing/index.md @@ -16,7 +16,7 @@ conflict with or break things for others. Azimuth supports using a single [configuration environment](../environments.md) to deploy multiple independent Azimuth instances. When -[activating an environment](../deployment/#activating-an-environment), a unique instance name +[activating an environment](../deployment/index.md#activating-an-environment), a unique instance name can be given as a second argument to the `activate` script, e.g.: ```bash diff --git a/docs/index.md b/docs/index.md index 96a8ea42..466bd489 100644 --- a/docs/index.md +++ b/docs/index.md @@ -11,7 +11,7 @@ driven by configuration derived from the [azimuth-config reference configuration](https://github.com/azimuth-cloud/azimuth-config). The `azimuth-config` repository is designed to be forked for a specific site and is structured -into multiple [environments](#environments). This structure allows common configuration to be +into multiple [environments](./environments.md). This structure allows common configuration to be shared but overridden where required using composition of environments. To try out Azimuth on your OpenStack cloud, you can follow [these instructions](./try.md) diff --git a/docs/repository/index.md b/docs/repository/index.md index 947fd001..c20e689e 100644 --- a/docs/repository/index.md +++ b/docs/repository/index.md @@ -103,7 +103,7 @@ component versions, upgraded dependencies and new images. If you have automated deployments, which is recommended for a production installation, this process - [can also be automated](../deployment/automation.md#automated-synchronisation-of-upstream-changes). + [can also be automated](../deployment/automation.md#automated-upgrades). To upgrade your Azimuth configuration to a new release, use the following steps to create a new branch containing the upgrade: diff --git a/mkdocs.yml b/mkdocs.yml index 98dcb8f2..7dc3d703 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -43,6 +43,9 @@ nav: - debugging/consul.md - debugging/kubernetes.md - debugging/caas.md + - Architecture: + - architecture/index.md + - architecture/identity.md - Developing: - developing/index.md @@ -67,7 +70,11 @@ markdown_extensions: anchor_linenums: true - pymdownx.inlinehilite - pymdownx.snippets - - pymdownx.superfences + - pymdownx.superfences: + custom_fences: + - name: mermaid + class: mermaid + format: !!python/name:pymdownx.superfences.fence_code_format - toc: toc_depth: 3 diff --git a/requirements-docs.txt b/requirements-docs.txt index a9c1f199..a9489709 100644 --- a/requirements-docs.txt +++ b/requirements-docs.txt @@ -1,3 +1,3 @@ -mkdocs==1.3.0 -mkdocs-material==8.3.8 +mkdocs==1.6.1 +mkdocs-material==9.5.42 mkdocs-git-revision-date-localized-plugin