Quarantine Http3RequestTests.GET_ConnectionsMakingMultipleRequests_AllSuccess #62908
Description
Failing Test(s)
- Interop.FunctionalTests.Http3.Http3RequestTests.GET_ConnectionsMakingMultipleRequests_AllSuccess
Error Message
System.Net.Http.HttpRequestException : An error occurred while sending the request.
---- System.Net.Http.HttpProtocolException : The HTTP/3 server closed the connection. HTTP/3 error code 'H3_INTERNAL_ERROR' (0x102). (HttpProtocolError)
Stacktrace
at System.Net.Http.Http3RequestStream.SendAsync(CancellationToken cancellationToken)
at System.Net.Http.Http3RequestStream.SendAsync(CancellationToken cancellationToken)
at System.Net.Http.Http3Connection.SendAsync(HttpRequestMessage request, WaitForHttp3ConnectionActivity waitForConnectionActivity, Boolean streamAvailable, CancellationToken cancellationToken)
at System.Net.Http.Http3Connection.SendAsync(HttpRequestMessage request, WaitForHttp3ConnectionActivity waitForConnectionActivity, Boolean streamAvailable, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.TrySendUsingHttp3Async(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.SocketsHttpHandler.<SendAsync>g__CreateHandlerAndSendAsync|115_0(HttpRequestMessage request, CancellationToken cancellationToken)
at Interop.FunctionalTests.Http3.Http3RequestTests.<GET_ConnectionsMakingMultipleRequests_AllSuccess>g__MakeRequest|15_2(HttpMessageInvoker client, String address, Int32 count) in /_/src/Servers/Kestrel/test/Interop.FunctionalTests/Http3/Http3RequestTests.cs:line 820
at Interop.FunctionalTests.Http3.Http3RequestTests.<GET_ConnectionsMakingMultipleRequests_AllSuccess>g__RunConnection|15_1(String address) in /_/src/Servers/Kestrel/test/Interop.FunctionalTests/Http3/Http3RequestTests.cs:line 808
at Interop.FunctionalTests.Http3.Http3RequestTests.GET_ConnectionsMakingMultipleRequests_AllSuccess() in /_/src/Servers/Kestrel/test/Interop.FunctionalTests/Http3/Http3RequestTests.cs:line 788
--- End of stack trace from previous location ---
----- Inner Stack Trace -----
Logs
The logs are too large to include in full inside the issue, but I created a gist. This part seems like the relevant bit:
| [8.075s] Microsoft.AspNetCore.Server.Kestrel.Transport.Quic Debug: Stream id "0HNEARDO098UB:0000011C" shutting down writes because: "The QUIC transport's send loop completed gracefully.".
| [8.075s] Microsoft.AspNetCore.Server.Kestrel.Transport.Quic Trace: Stream id "0HNEARDO098UB:0000011C" pooled for reuse.
| [8.093s] Microsoft.AspNetCore.Server.Kestrel.BadRequests Debug: Connection id "0HNEARDO098U7", Request id "0HNEARDO098U7:00000024": the connection was closed because the response was not read by the client at the specified minimum data rate.
| [8.094s] Microsoft.AspNetCore.Server.Kestrel.Http3 Debug: Connection id "0HNEARDO098U7": HTTP/3 connection error.
| Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http3.Http3ConnectionErrorException: HTTP/3 connection error (H3_INTERNAL_ERROR): The connection was timed out by the server because the response was not read by the client at the specified minimum data rate.
| [8.094s] Microsoft.AspNetCore.Server.Kestrel.Http3 Debug: Connection id "0HNEARDO098U7": GOAWAY stream ID 400.
| [8.094s] Microsoft.AspNetCore.Server.Kestrel.Http3 Trace: Connection id "0HNEARDO098U7" sending GOAWAY frame for stream ID 3 with length 2.
| [8.094s] Microsoft.AspNetCore.Server.Kestrel.Transport.Quic Debug: Connection id "0HNEARDO098U7" aborted by application with error code 258 because: "HTTP/3 connection error (H3_INTERNAL_ERROR): The connection was timed out by the server because the response was not read by the client at the specified minimum data rate.".
| [8.171s] Microsoft.AspNetCore.Server.Kestrel.BadRequests Debug: Connection id "0HNEARDO098U7", Request id "0HNEARDO098U7:00000028": the connection was closed because the response was not read by the client at the specified minimum data rate.
| [8.171s] Microsoft.AspNetCore.Server.Kestrel.Http3 Debug: Connection id "0HNEARDO098U7": HTTP/3 connection error.
https://gist.github.com/halter73/31c2659593fc55103de6804b5b36257f
We could disable rate limiting for this test, but I'm concerned this is a real bug caused by us bit too strict with the minimum response rate limit. 8 seconds for 1000 requests (100 per 10 parallel connections) does seem like a lot, but 8 seconds isn't too much more than the grace period.
With HTTP/1.1 and HTTP/2, we consider the size of the framing as part of what in HTTP/3 is transport-level (QUIC) when we calculate the data rate. I wonder if we need to account for this. @JamesNK