storages: s3 custom domain signature fix #17087
Conversation
S3 signatures are fickle since there are several hashes of components which must ultimately be correct for the signature to match with no room for error. Since both the *host* header as well as the path, the former by default, the latter necessarily. The issue with custom domains is therefore that the *host* header mismatches unless the signature was created for that specific domain. This is complicated further by *boto3* making it incredibly difficult to actually sign a link with the path *not* containing the bucket prefix. Therefore setting the endpoint alone does not work with a truly custom domain (i.e. one where the data resides on the root of the domain). Changing this to a rather hacky workaround accessing fields of the internal data structures and grabbing the signer directly allows for signing the request. This comes with several caveats such as the parameters going largely untouched, however in preliminary testing it works fine. There are no superfluous slashes (i.e. no `https://example.com//file.svg`), no superfluous prefixes (i.e. `https://example.com/example.com/example.com/file.svg`), and there are no signature errors in general. Uploads still work. Reminder that the application cache may need clearing before the changes reflect in the non-admin UI. Relates: - [issue 13463](https://redirect.github.com/goauthentik/authentik/issues/13463) - [pr 14706 ](https://redirect.github.com/goauthentik/authentik/pull/14706) Signed-off-by: benaryorg <[email protected]>
✅ Deploy Preview for authentik-storybook ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
❌ Deploy Preview for authentik-docs failed. Why did it fail? →
|
❌ Deploy Preview for authentik-integrations failed. Why did it fail? →
|
and
I'm not 100% sure, but the CI failures sound unrelated to this PR. |
|
I have confirmed this patch to work on both 2025.2.3 and 2025.8.3 on my end. Some of the files I uploaded intermediately (I think with a different patch) needed a reupload since their URLs had duplicate prefixes ( Removing the |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #17087 +/- ##
==========================================
+ Coverage 92.74% 92.79% +0.05%
==========================================
Files 794 845 +51
Lines 40424 45853 +5429
==========================================
+ Hits 37490 42550 +5060
- Misses 2934 3303 +369
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Details
Effectively this fixes s3 signatures for custom domains.
The solution for this is somewhat hacky, however it does apply cleanly on version/2025.2.4 and I have tested it and can confirm that it works.
I'll set this to draft state because I do not currently have the capacity to finish the testing and linting, so if anyone wants to pick this up, please go ahead.
To be perfectly honest, I just copied the stuff I use to generate such links personally straight out of my shell history into the source code.
It is however the only solution that I have managed to find which actually works.
NB: this has only been tested when cherry-picked onto 2025.2.3 and 2025.8.3!
NB: this is more or less a bandaid, a proper rewrite of the method to generate the URl would be desirable IMHO, especially since it will compute the signature even when it is not needed. Using the proper class for the URl would also be good, to avoid any weirdnesses for edge-cases. Right now the onus of providing a working string is put on the admin.
The commit message is included verbatim below.
S3 signatures are fickle since there are several hashes of components which must ultimately be correct for the signature to match with no room for error.
Since both the host header as well as the path, the former by default, the latter necessarily.
The issue with custom domains is therefore that the host header mismatches unless the signature was created for that specific domain.
This is complicated further by boto3 making it incredibly difficult to actually sign a link with the path not containing the bucket prefix.
Therefore setting the endpoint alone does not work with a truly custom domain (i.e. one where the data resides on the root of the domain).
Changing this to a rather hacky workaround accessing fields of the internal data structures and grabbing the signer directly allows for signing the request.
This comes with several caveats such as the parameters going largely untouched, however in preliminary testing it works fine.
There are no superfluous slashes (i.e. no
https://example.com//file.svg), no superfluous prefixes (i.e.https://example.com/example.com/example.com/file.svg), and there are no signature errors in general.Uploads still work.
Reminder that the application cache may need clearing before the changes reflect in the non-admin UI.
Relates:
Checklist
ak test authentik/)make lint-fix)If an API change has been made
make gen-build)If changes to the frontend have been made
make web)If applicable
make docs)