Support for payload signing + chunked encoding in DefaultAwsV4HttpSigner #6466

dagnir · 2025-10-07T21:17:48Z

Motivation and Context

Add support for payload signing of async streaming requests signed with SigV4 using default AwsV4HttpSigner (using AwsV4HttpSigner.create()). Note, requests using the http URI scheme will not be signed regardless of the value of AwsV4FamilyHttpSigner.PAYLOAD_SIGNING_ENABLED to remain consistent with existing behavior. This may change in a future release.

Modifications

Testing

Screenshots (if appropriate)

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)

Checklist

I have read the CONTRIBUTING document
Local run of mvn install succeeds
My code follows the code style of this project
My change requires a change to the Javadoc documentation
I have updated the Javadoc documentation accordingly
I have added tests to cover my changes
All new and existing tests passed
I have added a changelog entry. Adding a new entry must be accomplished by running the scripts/new-change script and following the instructions. Commit the new file created by the script in .changes/next-release with your changes.
My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

I confirm that this pull request can be released under the Apache 2 license

This commit adds support for SigV4 signing of async request payloads. In addition this commit moves the support for trailing checksums from HttpChecksumStage to the V4 signer implementation; this puts it in line with how sync chunked bodies are already handled.

- Rename to computeAndMoveContentLength - Rename variable to chunkedPayload - Fix javadocs - Sign payload only if non-null

Combine these two steps to improve performance by reducing memory allocations.

…coding-rebase

…coding-pt1

.changes/next-release/feature-AWSSDKforJavav2-e386e28.json

...aws/src/main/java/software/amazon/awssdk/http/auth/aws/internal/signer/util/SignerUtils.java

...ain/java/software/amazon/awssdk/http/auth/aws/internal/signer/AwsChunkedV4PayloadSigner.java

...c/main/java/software/amazon/awssdk/core/internal/http/pipeline/stages/HttpChecksumStage.java

...are/amazon/awssdk/http/auth/aws/internal/signer/chunkedencoding/ChunkedEncodedPublisher.java

...ava/software/amazon/awssdk/http/auth/aws/crt/internal/signer/AwsChunkedV4aPayloadSigner.java

...c/main/java/software/amazon/awssdk/core/internal/http/pipeline/stages/HttpChecksumStage.java

- Update variable naming - Throw if decoded content length not present - Throw if input request doesn't have content-length header

...va/software/amazon/awssdk/http/auth/aws/internal/signer/io/UnbufferedChecksumSubscriber.java

...a/software/amazon/awssdk/http/auth/aws/internal/signer/util/LengthCalculatingSubscriber.java

...aws/src/main/java/software/amazon/awssdk/http/auth/aws/internal/signer/util/SignerUtils.java

...ain/java/software/amazon/awssdk/http/auth/aws/internal/signer/AwsChunkedV4PayloadSigner.java

...va/software/amazon/awssdk/http/auth/aws/internal/signer/io/UnbufferedChecksumSubscriber.java

Fred1155 · 2025-10-15T17:24:20Z

Are we planning to have intergration test on the full S3 async flow for this PR? Or we'll add that after consolidating the behavior and before merging to master

dagnir · 2025-10-15T17:52:14Z

Are we planning to have intergration test on the full S3 async flow for this PR? Or we'll add that after consolidating the behavior and before merging to master

Did you have specific tests in mind?

We have existing integration tests that exercise the async checksumming: services/s3/src/it/java/software/amazon/awssdk/services/s3/checksum/AsyncHttpChecksumIntegrationTest.java

...c/main/java/software/amazon/awssdk/http/auth/aws/internal/signer/DefaultAwsV4HttpSigner.java

Fred1155 · 2025-10-15T19:06:27Z

Nice, we might need to update the TODO comments here:

aws-sdk-java-v2/services/s3/src/it/java/software/amazon/awssdk/services/s3/checksum/AsyncHttpChecksumIntegrationTest.java

Line 258 in 6f7f0e7

    
                * TODO: Update this test with right asserts when payload signing is supported in async.

dagnir · 2025-10-15T20:03:05Z

Nice, we might need to update the TODO comments here:

aws-sdk-java-v2/services/s3/src/it/java/software/amazon/awssdk/services/s3/checksum/AsyncHttpChecksumIntegrationTest.java

Line 258 in 6f7f0e7

* TODO: Update this test with right asserts when payload signing is supported in async.

Hmm that can't be addressed right now actually, same for the other TODO. AFAICT from the javadoc, it's expecting the body to be signed because the client is using an http endpoint. However, we are explicitly not forcing signing for http endpoints right now to preserve the existing behavior for async requests.

    /**
     * S3 clients by default don't do payload signing. But when http is used, payload signing is expected to be enforced. But
     * payload signing is not currently supported in async path (for both pre/post SRA signers).
     * However, this test passes, because of https://github
     * .com/aws/aws-sdk-java-v2/blob/38e221bd815af31a6c6b91557499af155103c21a/core/auth/src/main/java/software/amazon/awssdk/auth/signer/internal/AbstractAwsS3V4Signer.java#L279-L285.
     * Keeping this test enabled, to ensure moving to SRA Identity & Auth, does not break current behavior.
     * TODO: Update this test with right asserts when payload signing is supported in async.
     */

Fred1155 · 2025-10-15T20:11:39Z

Hmm that can't be addressed right now actually, same for the other TODO. AFAICT from the javadoc, it's expecting the body to be signed because the client is using an http endpoint. However, we are explicitly not forcing signing for http endpoints right now to preserve the existing behavior for async requests.

Oh you are right, I misunderstood what this test does.

…sra-chunk-encoding-pt1

sonarqubecloud · 2025-10-16T18:05:51Z

Quality Gate passed

Issues
32 New issues
0 Accepted issues

Measures
0 Security Hotspots
93.1% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

github-actions · 2025-10-16T18:53:40Z

This pull request has been closed and the conversation has been locked. Comments on closed PRs are hard for our team to see. If you need more assistance, please open a new issue that references this one.

dagnir added 5 commits August 5, 2025 14:14

Review comments

a222719

- Rename to computeAndMoveContentLength - Rename variable to chunkedPayload - Fix javadocs - Sign payload only if non-null

Combine chunking + encoding

1d15408

Combine these two steps to improve performance by reducing memory allocations.

Merge remote-tracking branch 'origin/master' into dongie/sra-chunk-en…

3d1d896

…coding-rebase

Merge remote-tracking branch 'origin/master' into dongie/sra-chunk-en…

119cfbf

…coding-pt1

dagnir force-pushed the dongie/sra-chunk-encoding-pt1 branch from fcda677 to 488b723 Compare October 7, 2025 22:18

Fixes after merge

e52ea62

dagnir force-pushed the dongie/sra-chunk-encoding-pt1 branch from 3a9cb34 to e52ea62 Compare October 8, 2025 21:30

Merge branch 'master' into dongie/sra-chunk-encoding-pt1

e1df197

dagnir marked this pull request as ready for review October 8, 2025 21:40

dagnir requested a review from a team as a code owner October 8, 2025 21:40

RanVaknin reviewed Oct 8, 2025

View reviewed changes

.changes/next-release/feature-AWSSDKforJavav2-e386e28.json Show resolved Hide resolved

Add store test for AsyncSigningStage

7cd278b

zoewangg reviewed Oct 10, 2025

View reviewed changes

dagnir mentioned this pull request Oct 10, 2025

Support async V4 payload signing #6314

Closed

12 tasks

zoewangg reviewed Oct 10, 2025

View reviewed changes

...ava/software/amazon/awssdk/http/auth/aws/crt/internal/signer/AwsChunkedV4aPayloadSigner.java Show resolved Hide resolved

RanVaknin reviewed Oct 10, 2025

View reviewed changes

...c/main/java/software/amazon/awssdk/core/internal/http/pipeline/stages/HttpChecksumStage.java Show resolved Hide resolved

dagnir added 3 commits October 10, 2025 12:22

Review comments

0a236dc

- Update variable naming - Throw if decoded content length not present - Throw if input request doesn't have content-length header

Additional test for no content-length

d480132

Fix test after throw change

977d738

RanVaknin reviewed Oct 13, 2025

View reviewed changes

...va/software/amazon/awssdk/http/auth/aws/internal/signer/io/UnbufferedChecksumSubscriber.java Show resolved Hide resolved

RanVaknin reviewed Oct 13, 2025

View reviewed changes

...a/software/amazon/awssdk/http/auth/aws/internal/signer/util/LengthCalculatingSubscriber.java Outdated Show resolved Hide resolved

Remove unneeded class

acc4e6c

RanVaknin approved these changes Oct 13, 2025

View reviewed changes

dagnir changed the base branch from master to feature/master/sra-async-chunked-encoding October 14, 2025 20:15

zoewangg reviewed Oct 15, 2025

View reviewed changes

Fred1155 reviewed Oct 15, 2025

View reviewed changes

...c/main/java/software/amazon/awssdk/http/auth/aws/internal/signer/DefaultAwsV4HttpSigner.java Show resolved Hide resolved

Fred1155 approved these changes Oct 15, 2025

View reviewed changes

Review comments

efa994b

dagnir force-pushed the dongie/sra-chunk-encoding-pt1 branch from 2da656f to efa994b Compare October 15, 2025 20:36

zoewangg approved these changes Oct 15, 2025

View reviewed changes

dagnir and others added 2 commits October 15, 2025 15:30

Merge branch 'feature/master/sra-async-chunked-encoding' into dongie/…

3ea10dd

…sra-chunk-encoding-pt1

Update ChecksumSubscriber javadoc

118c85f

dagnir merged commit 5cbd30a into feature/master/sra-async-chunked-encoding Oct 16, 2025
39 of 40 checks passed

github-actions bot locked as resolved and limited conversation to collaborators Oct 16, 2025

Support for payload signing + chunked encoding in DefaultAwsV4HttpSigner #6466

Support for payload signing + chunked encoding in DefaultAwsV4HttpSigner #6466

Uh oh!

Conversation

dagnir commented Oct 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation and Context

Modifications

Testing

Screenshots (if appropriate)

Types of changes

Checklist

License

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Fred1155 commented Oct 15, 2025

Uh oh!

dagnir commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Fred1155 commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dagnir commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Fred1155 commented Oct 15, 2025

Uh oh!

sonarqubecloud bot commented Oct 16, 2025

Quality Gate passed

Uh oh!

Uh oh!

github-actions bot commented Oct 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

dagnir commented Oct 7, 2025 •

edited

Loading

dagnir commented Oct 15, 2025 •

edited

Loading

Fred1155 commented Oct 15, 2025 •

edited

Loading

dagnir commented Oct 15, 2025 •

edited

Loading