Keycloak auth manager causes long loading times for the webserver

[cboettcher]

After configuring the Keycloak auth manager (from the keycloak providers package) in our helm-based deployment, i realized that the loading times of the website were incredibly high.

I then did some digging through the logs, and found that the auth manager seems to send a new request to keycloak for every kind of permission that was needed, causing between 2 and 4 requests per second for a single user.

Since this seemed too much for me, i looked through the code of the auth manager and found that indeed, unless a batch_permission was checked, it would create a new request every time.

Is there a specific reason for this, instead of requesting and caching the whole set of permissions at once? The python-keycloak library offers functions for retrieving all permissions, so even short-lived caches would reduce the amount of requests to keycloak significantly.

I don't know if this is considered a bug, so i created this discussion thread. I'm also willing to create an issue instead, but wanted to get a second opinion first.

[jhgoebbert]

cc @vincbeck WDYT?

[cboettcher]

Some additional context from what I've already looked into and thought about:

Here Is the function that is called for every singel-resource authentication request, and it creates a request to keycloak every time. The batch function does not seem to be used much.

---

I tried something like this, which seemed to solve the loading time issue, but it kinda ignores the context attributes and has no proper caching.

class KeycloakAuthManagerPermissions():
    """A set of permissions."""

    def __init__(self, permissions: dict[str, list[str]]) -> None:
        self.permissions = permissions

    def get_permissions(self) -> dict[str, list[str]]:
        return self.permissions
    
    def get_resource_permissions(self, resource_name: str) -> list[str]:
        return self.permissions.get(resource_name, [])
    
    @staticmethod
    def get_permissions_from_keycloak_string(keycloak_permissions: list[dict[str, str | list[str]]]) -> KeycloakAuthManagerPermissions:
        formatted_permissions: dict[str, list[str]] = {}

        for single_permission in keycloak_permissions:
            resource_name = single_permission.get("rsname")
            allowed_method_list = single_permission.get("scopes", [])
            if resource_name and isinstance(resource_name, str) and isinstance(allowed_method_list, list):
                formatted_permissions[resource_name] = allowed_method_list

        return KeycloakAuthManagerPermissions(formatted_permissions)

    def _get_user_perms(
            self,
            *,
            user: KeycloakAuthManagerUser
    ) -> KeycloakAuthManagerPermissions:
        #TODO store and cache permissions token properly instead of this dict

        if self._cached_perms.get(user.get_id()):
            log.debug(f"Using cached permissions for user {user.get_name()}")
        else:
            kc_client = self.get_keycloak_client()
            permissions = kc_client.uma_permissions(token=user.access_token)
            log.debug(f"Retrieved fresh permissions of user {user.get_id()}.")
            
            self._cached_perms[user.get_id()] = KeycloakAuthManagerPermissions.get_permissions_from_keycloak_string(permissions)
        

        return self._cached_perms[user.get_id()]

def _is_authorized(
        self,
        *,
        method: ResourceMethod | str,
        resource_type: KeycloakResource,
        user: KeycloakAuthManagerUser,
        resource_id: str | None = None,
        attributes: dict[str, str | None] | None = None,
    ) -> bool:
        permissions = self._get_user_perms(user=user)

        context_attributes = prune_dict(attributes or {})
        if resource_id:
            context_attributes[RESOURCE_ID_ATTRIBUTE_NAME] = resource_id
        elif method == "GET":
            method = "LIST"

        log.debug(f"Trying to get access to {resource_type.value}-{method} for user {user.get_name()}")

        return method in permissions.get_resource_permissions(resource_type.value)

[vincbeck]

Hey, thanks for the feedback and all the details, that's very helpful. There is one open PR to address the same issue I think: #56433. I plan to look at that PR this week. It seems there is indeed a performance issue with Keycloak and I'll fix it anytime soon. In the meantime, if you can look at the PR I just linked and provide feedbacks whether you think it would resolve your issue