HTTP client / Guides

Features

Everything ioxide.httpclient does, what each option changes, and what it does at the edges.

Names and DNS#

HttpClientOptions.Host takes a name, an IPv4 address or an IPv6 address (with or without brackets). An address is dialed as it is. A name is resolved by the system resolver - the hosts file, resolv.conf, and whatever caching the system does - each time a connection opens:

• The lookup runs off the reactor thread, through Dns.GetHostAddressesAsync, and its result is posted back to the reactor. It never happens per request: an open connection is reused without resolving anything.
• Every address the resolver returns, IPv4 and IPv6, is tried in its order - the system's own preference - moving to the next when one refuses or times out.
• A name that does not resolve fails the open with could not resolve <name>: <the resolver's reason>, and a request that times out waiting says so: no connection to <host>:<port> within <n> ms: could not resolve ....

Because every new connection resolves again, a pool follows a name that moves as its connections are replaced. A pool whose connections stay busy forever never replaces them; MaxLifetimeMs is for that.

IPv6#

A connection's socket takes the family of the address it dials, so an IPv6 address - given in Host, or returned by the resolver - is reached over IPv6, and a link-local address keeps its scope. In the request's host header an IPv6 address is written in brackets: host: [::1]:8080.

The host header#

The client writes host on every request, from Host as you gave it - so a name goes out as a name, which is what a server hosting many sites under one address routes by. The port is added unless it is the scheme's own: 80 for http, 443 for https. Do not add a host header yourself: a second one makes the request invalid, and servers must refuse it.

TLS#

Set HttpClientOptions.Tls to a TlsClientContext for an https origin. The context is the configured BoringSSL client context; one serves any number of connections and origins, and the code that creates it disposes it. RingHttpClient makes one of its own for all its https URLs.

The client keeps TLS in user space with BoringSSL - no kernel TLS - and holds no session cache, so every new connection does a full handshake.

Connect timeout and address fallback#

ConnectTimeoutMs (10 s) bounds how long one address may take to accept the connection. It is the kernel that enforces it: the socket's TCP_USER_TIMEOUT is set before the connect, so the kernel stops retrying the SYN and the connect fails with connect to <address> timed out after <n> ms; it is cleared once connected, so it does not apply to the connection afterwards. The first SYN retransmission comes after a second, so a timeout below a second acts at about a second.

A refused connect fails at once with connect to <address> failed: errno 111. Either way the next address is tried, and the last failure is what the open reports. A TLS handshake that fails is not retried on another address: the origin was reached, and refused.

Keep-alive and reuse#

A connection serves one request at a time - no pipelining - and goes back to the pool as soon as its response is read. The pool hands out the most recently used idle connection first, which keeps the set of warm connections small and lets the oldest idle ones go. A connection is closed instead of reused when:

• the response says Connection: close, or is HTTP/1.0 without Connection: keep-alive;
• the response had no framing header, so its body ran until the origin closed;
• a send, a receive or the parser failed;
• it is older than MaxLifetimeMs.

Connections the origin closed#

An origin ends idle connections - a keep-alive timeout is the most ordinary way an upstream connection ends. Nothing reads an idle pooled connection, so its FIN would wait unread until the next request was written into it and failed. The sweep checks each idle connection every SweepIntervalMs with a poll() for POLLRDHUP and replaces the ones that are closed or reset. It works over TLS too, where a close_notify and a session ticket can sit in front of the FIN. A close that lands between two sweeps can still fail a request, as it would without the sweep; nothing retries it. See the sweep.

Idle timeout and lifetime#

The idle timeout is client-side: with it at 135 s, origins that time out sooner - nginx at 75 s, Kestrel at 130 s, many at 5 s - still close first, and the sweep catches it.

Pool size, opens and backoff#

PoolSize is the most connections a pool keeps to its origin, opened and opening together, and so the most requests in flight to it from one reactor. HttpClientPool.Start opens all of them up front and the sweep keeps it there. A request that finds no idle connection opens one if there is room, so a burst opens in parallel, up to PoolSize. After a failed connect, new opens wait 200 to 400 ms, picked at random, so a pool facing a dead origin does not hammer it.

When every connection is busy, requests wait in order. AcquireTimeoutMs (10 s) bounds the wait: past it the request fails with no connection to <host>:<port> within <n> ms, followed by why the last connect failed, when it did. The sweep checks the deadline every tick, so it holds to within 250 ms.

Responses#

• Framing. Content-Length, chunked transfer encoding, or - with neither - the body runs until the origin closes, and the connection is not reused. A response to HEAD, a 1xx, a 204 and a 304 have no body whatever their headers say: a HEAD response's Content-Length is the size the body would have had, and reading it as one would take the next response for this one's body.
• Interim responses. 100 Continue and 103 Early Hints are read and skipped; the response you get is the final one.
• Chunked bodies are joined into one body; chunk extensions and trailers are read and dropped.
• Size. MaxResponseBytes (8 MiB) caps the head and body together: a larger response fails the request instead of growing without bound, and the header block is held to the same ceiling. A response is buffered whole - there is no streaming.
• Buffers. Each connection has a send buffer and a receive buffer (SendBufferSize, ReceiveBufferSize, 16 KiB each). The receive buffer grows when a response needs more, up to MaxResponseBytes; a request body larger than the send buffer is sent from your memory directly.
• The response owns a pooled copy of its headers and body, header names lowercased, so the connection is reused the moment it is read. Dispose it to return the buffer.

Hardening#

Response heads are parsed by Glyph11's hardened parser, which refuses what would let a proxy and an origin disagree about where a response ends, and the client treats any refusal as the end of that connection:

• Transfer-Encoding and Content-Length together, and two Content-Length headers that disagree;
• a header line folded onto the next (obs-fold), whitespace before a colon, and a bare LF inside a header line;
• characters a header name or value may not hold.

A refusal fails the request with malformed response head: <the rule broken>. Past the head:

• Chunk sizes are strict. A size is hexadecimal digits and nothing else, no larger than int.MaxValue, optionally followed by whitespace and a ; extension. Anything else fails the response with malformed chunk size, and a chunk whose data runs past its size with malformed chunk: data runs past its size.
• Bytes past a response's end retire its connection. Nothing is pipelined, so a byte after a complete response could only be misread as the next one: the response is returned, and the pool discards the connection and opens a replacement.
• Request heads are checked. A CR, LF or NUL in the method, the target, a header name or a header value throws ArgumentException - request <part> contains CR, LF or NUL - before a connection is taken, so nothing is written and the pool loses nothing.

Requests by URL#

RingHttpClient takes absolute http:// and https:// URLs, parsed by System.Uri; anything else throws ArgumentException. The request target sent is the URL's path and query, as Uri normalizes them; the fragment is not sent. An international name is resolved in its ASCII form, and http://[::1]:8080/ reaches an IPv6 address.

• A pool per origin - per scheme, host and port - made on the first request to it, from RingHttpClientOptions.Pool: every option there applies to every origin except Host, Port and Tls, which come from the URL.
• On demand. An origin's pool opens nothing up front: connections open as requests need them, in parallel for a burst, up to PoolSize (64 by default), and close once idle past IdleTimeoutMs. A pool with nothing open, opening or inside it is dropped by the next sweep, so the client keeps nothing for an origin it no longer speaks to.
• One TLS context for every https origin, each connection checked against its URL's host - from RingHttpClientOptions.Tls, or the system trust store with ALPN http/1.1 when that is not set. Its ServerName is ignored.

Redirects#

RingHttpClient follows redirects the way .NET's HttpClient does:

• A relative Location is resolved against the URL that answered.
• A redirect from https to http is never followed: the redirect is returned as the response.
• authorization and cookie headers are kept while the redirects stay within the origin, and dropped for the rest of the chain once one leaves it.
• Past MaxRedirects (50, as .NET) the last redirect is returned as the response; 0 follows none. Each response passed over is disposed.

HttpClientPool follows nothing: a redirect is a response like any other.

Errors#

A failure throws HttpClientException from the request, with a message that says what went wrong; a connection the failure happened on is closed. The messages you will see: