Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed stream request hangs until timeout #1973

Open
reimai opened this issue Jul 31, 2024 · 0 comments
Open

Failed stream request hangs until timeout #1973

reimai opened this issue Jul 31, 2024 · 0 comments

Comments

@reimai
Copy link

reimai commented Jul 31, 2024

Hi, I'm using async-http-client 2.12.3 for stream requests. Suddenly all failed requests are cancelled by timeout (1m).
Debugging the issue shows that these requests are actually failing almost immediately due to a closed channel, but unsuccessfully recovered with a warning of a closed stream to time out a minute later:

2024-07-30 17:56:25.809 | zio-default-blocking-35   org.asynchttpclient.netty.request.NettyRequestSender   Using pooled Channel '[id: 0x64b880e7, L:/[my-client-ip]:56122 - R:my-server/[my-ip]:8443]' for 'POST' to my server

2024-07-30 17:56:25.809 | zio-default-blocking-35   org.asynchttpclient.netty.request.NettyRequestSender   Using open Channel [id: 0x64b880e7, L:/[my-client-ip]:56122 - my-sever/[my-ip]:8443] for POST 

2024-07-30 17:56:25.898 | AsyncHttpClient-4-1   org.asynchttpclient.netty.request.WriteListener   Write exception on pooled channel, letting retry trigger    stack_trace: java.nio.channels.ClosedChannelException: null

2024-07-30 17:56:25.899 | AsyncHttpClient-4-1   org.asynchttpclient.netty.handler.HttpHandler   Channel Closed: [id: 0x64b880e7, L:/[my-client-ip]:56122 ! my-server/[my-ip]:8443] with attribute NettyResponseFuture{currentRetry=0, 	isDone=0, 	isCancelled=0, 	asyncHandler=sttp.client3.asynchttpclient.AsyncHttpClientBackend$$anon$1@22721d20, 	nettyRequest=org.asynchttpclient.netty.request.NettyRequest@5f78e6db, 	future=java.util.concurrent.CompletableFuture@481ada03[Not completed], 	uri=my-server:8443?query=INSERT+INTO+profiler_collector.samples+(mode,+service_name,+timestamp,+labels.key,+labels.value,%0A++stack_trace.method_id,+stack_trace.line,+stack_trace.type,+samples)%0AVALUES%0A, 	keepAlive=true, 	redirectCount=0, 	timeoutsHolder=org.asynchttpclient.netty.timeout.TimeoutsHolder@1bfcc8d7, 	inAuth=0, 	touch=1722351385898}

2024-07-30 17:56:25.899 | AsyncHttpClient-4-1   org.asynchttpclient.netty.request.NettyRequestSender   Trying to recover request DefaultHttpRequest(decodeResult: success, version: HTTP/1.1)

2024-07-30 17:56:25.907 | AsyncHttpClient-4-3   org.asynchttpclient.netty.channel.NettyConnectListener   Using new Channel '[id: 0xba9d5427, L:/[my-client-ip]:57472 - my-server/[my-ip]:8443]' for 'POST' to my-server

2024-07-30 17:56:25.908 | AsyncHttpClient-4-3   org.asynchttpclient.netty.request.body.NettyReactiveStreamsBody   Stream has already been consumed and cannot be reset

2024-07-30 17:57:25.932 | AsyncHttpClient-timer-3-1   org.asynchttpclient.netty.request.NettyRequestSender   Request timeout to my-server/my-ip:8443 after 60000 ms 

Setting maxRetries to 0 and retrying requests outside of async-http-client solves the issue, requests are retried immediately and successfully.
Should NettyReactiveStreamsBody just log a warning for an already consumed stream? Why would not it cancel the response Future and maybe close the Channel, so the user would not wait for a timeout?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant