Timeouts and HTTP Connection Reuse

10 May, 2026

When building an application which talks to some dependencies over the network it's generally recommended to always specify timeouts¹. This prevents resources (threads, connections, descriptors) from being occupied for a long time — worst case indefinitely² - when a dependency is slow or unavailable, leading to the application becoming unresponsive. Timeouts also allow retrying by contacting a different server instead of waiting for a response from a struggling machine that might never arrive.

Timeouts can be configured on different levels: establishing a connection, the whole request-response cycle, awaiting a response, or individual IO operations. Connection timeouts can be safely retried and are relatively straightforward to reason about. Hitting a timeout when performing a request is more tricky — this can result in an unusable connection. Let's look at the details to understand why exactly this is the case, using HTTP as an example, keeping in mind that the same issues arise when working with other network protocols.

Let's look at HTTP/1.1 first before exploring how HTTP/2 and HTTP/3 improve upon it. The underlying transport protocol used is TCP. A TCP connection is a bidirectional stream. It can be thought of as a pair of channels, one for sending data (request) and one for receiving (responses), each a sequence of bytes delivered reliably and in-order. Programs interact with a connection represented as a socket (an operating system API) that can be written to (send()) and read from (recv())³.

To avoid paying the latency cost of establishing a connection for each request, a connection is reused to send/receive multiple requests/responses (messages). A pool of connections is typically maintained to perform more than one request concurrently. The two most common approaches for distinguishing messages sent over the same socket from one another are delimiters and length-prefixing⁴. With delimiters a sequence of bytes is acting as a separator (e.g. a newline separates HTTP message headers from message body and the next message). With length-prefixing the size of the payload is specified at the beginning of the message (HTTP Content-Length header and chunked encoding are variations of this approach⁵).

Depending when a timeout occurs during the request-response exchange, the connection is left in a different state. First the request data is queued up for transmission by handing it over to the OS kernel using send(), which might be accepted in chunks, requiring multiple function calls. At this point we don't know yet if the data was actually sent out or delivered⁶, reflecting the asynchronous nature of the network. When the network is congested or the server is overloaded, a timeout might occur before all request data had been queued up. A connection in this state cannot be used to send a retry or any other request, as this would lead to data corruption due to broken message boundaries.

The below diagram illustrates a situation when a message prefix (yellow) was not written in full and a retry attempt (green) leads to broken message boundaries. In the first example a delimiter (\r\n) between messages is missing and in the second the length prefix (14:) is misinterpreted as a part of the previous message.

Most often a timeout occurs when waiting for a response or a part of the response to arrive. If reading of the remaining data from the socket is abandoned, the next read will receive the tail of the previous message when expecting a beginning of the next message.

Here reusing the connection to perform a different request results in a fragment of the previous response (red) read from the socket when a new response (green) is expected.

Given TCP guarantees that data is eventually delivered and delivered in order, it's theoretically possible to avoid corruption by making sure requests/responses are always sent/received completely, even after a timeout occurs. This however would increase complexity and force both the server and the client to do additional work only to throw the results away. Also there's no guarantee that it would be possible to recover a connection, potentially already abandoned by the server — the process would need its own timeout. And, finally, HTTP allows message bodies without a predetermined length using chunked encoding and connection bound responses⁷. In practice, HTTP clients sidestep these issues altogether by having a blanket policy to close a connection on a timeout, as well as on an IO error sending/receiving data.

Establishing a new connection introduces an additional delay compared to a typical request-response time. Use of TLS to protect the data in-flight has a cost: 2 RTT (HTTP/1.1 and HTTP/2) and CPU-intensive cryptographic work for the server. To amortize the cost of opening a connection, pooling can be used, which allows connections to be established ahead of time and kept open when not in use. Pooling only helps if a free connection is available - a sequence of retries, especially happening across multiple concurrent requests, can quickly lead to replacing all the connections, exhausting the pool. Reestablishing multiple connections, all at once and potentially from many clients, when the server is already struggling with load would only make the situation worse. Therefore, you probably don't want too tight timeouts and too eager retries. Timeouts should be rather rare and not something that happens regularly in response to every variation in response latency.

HTTP/2 introduces stream multiplexing over a single TCP connection: messages are not sent sequentially but are broken down into interleaved frames, each belonging to a stream. Length-prefixed frames within each stream are processed sequentially and are reassembled into HTTP request/response messages by the receivers. Streams can be closed without terminating the underlying connection. A timeout in the middle of sending/receiving a message would require closing of the corresponding stream, but only a partially written/received frame would make the whole TCP connection unusable. As each frame has a known size, it's also easier to bring a connection to a safe state after a timeout by finishing reading/writing of the frame before resetting the stream, compared to HTTP/1.x.

HTTP/3 is similar to HTTP/2 in that it multiplexes streams over a single underlying transport connection. A significant difference is that the underlying transport protocol is QUIC and not TCP. QUIC itself provides multiplexing - a stream for each HTTP/3 request/response pair. Even if an HTTP/3 frame is written to a QUIC stream partially, it doesn't affect the other streams and the underlying connection⁸. QUIC achieves this by delivering each frame within a single UDP datagram. Partial reads are avoided as UDP datagrams are transferred between a program and an OS kernel as one piece of data⁹ - reading from a UDP socket (recvmsg()) will yield a complete QUIC frame. On top of UDP QUIC cryptographically ensures packet integrity by integrating TLS.

To sum it up, client timeouts are important for resiliency and resource utilization. With a stream-oriented transport protocol such as TCP, timeouts can break message framing due to partial reads/writes that prevents connection reuse. This affects application protocols like HTTP/1.x and HTTP/2. A typical client implementation will close the connection whenever a timeout occurs. Too tight request timeouts can negatively impact latency and cause connection pool exhaustion. HTTP/3 is based on QUIC, instead of TCP, which addresses these issues by building on UDP, a message oriented protocol, allowing it to maintain framing for concurrent independent streams within the same connection, even in the face of partially sent/received messages.

Updated 2026.08.07