ioxidedocs

HTTP client Guides

Recipes

Working shapes for the common jobs, and how to size a pool for a reverse proxy.

A reverse proxy#

An HTTP/1.1 front end on port 8080 that forwards each request's target to a backend and passes the answer back. One pool per reactor, so the call out and the answer back stay on the thread the request came in on. The target is cut at the request line's first CR or LF, so nothing a client sends can add lines to the request the backend sees.

using System.Text;
using ioxide;
using ioxide.httpclient;
using ioxide.utils;

var config = new ServerConfig { ReactorCount = Environment.ProcessorCount, Tcp = new TcpOptions { Port = 8080 } };
var backend = new HttpClientOptions
{
    Host          = "backend.internal",
    Port          = 9000,
    PoolSize      = 128,       // at least the requests one reactor has in flight to it
    MaxLifetimeMs = 300_000,   // re-resolve every five minutes, so a backend that moves is followed
};

for (int id = 0; id < config.ReactorCount; id++)
{
    var reactor = new Reactor(id, config);
    reactor.OnStart = r => HttpClientPool.Start(r, backend);
    reactor.TcpHandle = async (r, conn) =>
    {
        HttpClientPool pool = r.GetService<HttpClientPool>();
        try
        {
            while (true)
            {
                RecvSnapshot snapshot = await conn.ReadAsync();
                string? target = null;
                bool received = false;
                while (conn.TryGetItem(snapshot, out SpscRecvRing.Item item))
                {
                    if (item.HasBuffer)
                    {
                        target ??= RequestTarget(item.AsSpan());
                        received = true;
                        conn.ReturnBuffer(in item);
                    }
                }

                if (received)
                {
                    await ForwardAsync(conn, pool, target);
                }
                if (snapshot.IsClosed) return;
                conn.ResetRead();
            }
        }
        finally
        {
            conn.DecRef();
        }
    };
    new Thread(reactor.Run) { Name = $"reactor-{id}" }.Start();
}

static async ValueTask ForwardAsync(TcpConnection conn, HttpClientPool pool, string? target)
{
    if (target is null)
    {
        conn.Write("HTTP/1.1 400 Bad Request\r\nContent-Length: 0\r\n\r\n"u8);
    }
    else try
    {
        using HttpClientResponse response = await pool.GetAsync(target);
        string type = response.TryGetHeader("content-type"u8, out ReadOnlyMemory<byte> t)
            ? Encoding.ASCII.GetString(t.Span) : "application/octet-stream";
        conn.Write(Encoding.ASCII.GetBytes(
            $"HTTP/1.1 {response.Status} OK\r\nContent-Type: {type}\r\nContent-Length: {response.Body.Length}\r\n\r\n"));
        conn.Write(response.Body.Span);
    }
    catch (HttpClientException e)
    {
        byte[] message = Encoding.ASCII.GetBytes($"upstream: {e.Message}");
        conn.Write(Encoding.ASCII.GetBytes($"HTTP/1.1 502 Bad Gateway\r\nContent-Length: {message.Length}\r\n\r\n"));
        conn.Write(message);
    }
    await conn.FlushAsync();
}

// "GET /path HTTP/1.1" -> "/path", from the request line alone.
static string? RequestTarget(ReadOnlySpan<byte> request)
{
    int end = request.IndexOfAny((byte)'\r', (byte)'\n');
    ReadOnlySpan<byte> line = end < 0 ? request : request[..end];
    int first = line.IndexOf((byte)' ');
    int last = line.LastIndexOf((byte)' ');
    if (first < 0 || last <= first + 1 || line[first + 1] != (byte)'/' || line.Contains((byte)0)) return null;
    return Encoding.ASCII.GetString(line[(first + 1)..last]);
}

It forwards GETs and answers with the backend's status, content type and body; a real proxy parses the whole request and forwards its method, headers and body too. The repository's Playground/Proxy/H1ToH1, H2ToH1 and H3ToH1 samples do the same from HTTPS, HTTP/2 and HTTP/3 front ends.

Several backends#

A gateway that sends each path to its own backend starts one pool per backend on every reactor, each with its own options. Start registers the pool as a reactor service, and services are keyed by type: a second Start on the same reactor replaces the first in GetService<HttpClientPool>(), though both keep running. So keep the pools Start returns, in a service of your own:

// One reactor's backends by path prefix; the first that matches wins.
sealed class Backends((string Prefix, HttpClientPool Pool)[] routes)
{
    public HttpClientPool? For(string target)
    {
        foreach ((string prefix, HttpClientPool pool) in routes)
        {
            if (target.StartsWith(prefix, StringComparison.Ordinal)) return pool;
        }
        return null;
    }
}

reactor.OnStart = r => r.AddService(new Backends(
[
    ("/users/",  HttpClientPool.Start(r, new HttpClientOptions { Host = "users.internal",  Port = 9000, PoolSize = 64 })),
    ("/orders/", HttpClientPool.Start(r, new HttpClientOptions { Host = "orders.internal", Port = 9000, PoolSize = 16, IdleTimeoutMs = 0 })),
]));

// in the handler, where the proxy above takes its one pool
if (r.GetService<Backends>().For(target) is HttpClientPool pool)
{
    using HttpClientResponse response = await pool.GetAsync(target);
    // ... answer with it, as above
}
else
{
    // no backend for this path: answer 404
}

Each pool keeps its own connections, options and sweep, so a backend that goes down fails only its own routes, and IdleTimeoutMs = 0 keeps the orders backend's idle connections until it closes them. With many backends, a dictionary inside the holder works the same way. For origins not known in advance, RingHttpClient opens a pool per origin on first use - at the cost of parsing a URL on every request, opening connections only as requests need them, and one set of options for every origin. The repository's Playground/Clients/Gateway sample is this gateway, whole.

Sizing a pool for a proxy#

  • PoolSize is per reactor. Give it at least as many connections as one reactor has requests in flight to the backend. A request that finds every connection busy waits - cheaply, for 0.4 us of CPU, but it waits - and past AcquireTimeoutMs it fails.
  • ReceiveBufferSize: a response larger than the buffer grows it, and a connection's buffer stays at the size it grew to. Set it near the size of a typical response and leave MaxResponseBytes as the ceiling.
  • IdleTimeoutMs at or below the backend's own keep-alive timeout means the proxy closes idle connections before the backend does, rather than finding them closed.
  • MaxLifetimeMs when the backend is a name whose addresses change - a service behind DNS, a blue-green switch.
  • Leave SweepIntervalMs at 250 ms unless a pool is very large: the sweep costs a poll() per idle connection, and nothing per request.

Calling a JSON API by URL#

using System.Text.Json;

RingHttpClient http = reactor.GetService<RingHttpClient>();

var request = new HttpClientRequest(HttpMethods.Get, "/");
request.Headers.Add("accept"u8.ToArray(), "application/json"u8.ToArray());
request.Headers.Add("user-agent"u8.ToArray(), "my-service/1.0"u8.ToArray());

using HttpClientResponse response = await http.SendAsync("https://api.github.com/repos/MDA2AV/ioxide", request);
if (response.Status == 200)
{
    using JsonDocument repo = JsonDocument.Parse(response.Body);
    Console.WriteLine(repo.RootElement.GetProperty("stargazers_count").GetInt32());
}

The body is UTF-8 bytes, so it goes to the JSON reader without a string in between. Parse it before the response is disposed.

Sending JSON#

var post = new HttpClientRequest(HttpMethods.Post, "/")
{
    Body = JsonSerializer.SerializeToUtf8Bytes(new { name = "ioxide" }),
};
post.Headers.Add("content-type"u8.ToArray(), "application/json"u8.ToArray());

using HttpClientResponse created = await http.SendAsync("https://api.example.com/items", post);

An origin with a private CA#

Trust your CA rather than turning verification off. One context per reactor, shared by that reactor's pool:

reactor.OnStart = r => HttpClientPool.Start(r, new HttpClientOptions
{
    Host = "billing.internal",     // also the name the certificate is checked against
    Port = 443,
    Tls  = TlsClientContext.Create(new TlsClientOptions
    {
        CaFile        = "/etc/ioxide/internal-ca.pem",
        AlpnProtocols = ["http/1.1"],
    }),
});

To dial an address but check the certificate against a name - a backend reached by IP whose certificate names a service - set ServerName. The host header still carries Host. When the certificate was issued for the address itself, set ServerName to the address: it is checked against the certificate's IP address entries, and no SNI is sent.

Mutual TLS#

Tls = TlsClientContext.Create(new TlsClientOptions
{
    CertificateFile = "/etc/ioxide/client.pem",   // leaf first, then its intermediates
    PrivateKeyFile  = "/etc/ioxide/client.key",
    CaFile          = "/etc/ioxide/internal-ca.pem",
    AlpnProtocols   = ["http/1.1"],
    MinimumVersion  = OpenSslVersions.Tls13,
}),

An origin that requires a client certificate and gets none refuses the handshake, and the request fails with the handshake's error.

Following an origin that moves#

A pool resolves its name each time it opens a connection. Connections that stay busy are never reopened, so on a busy pool set MaxLifetimeMs: each connection is replaced once it reaches that age - after its request, never during it - and the replacement resolves the name again. Five minutes is a reasonable start; the cost is one new connection per connection per lifetime.

Testing against a self-signed origin#

Point CaFile at the test certificate - a self-signed certificate is its own root. VerifyCertificate = false also works, and accepts any certificate from anyone, so keep it out of anything but a test.