HTTP/2

HTTP/2

Two HTTP/2 servers for ioxide: ioxide.http2, written in C# with its own HPACK, and ioxide.nghttp2, with the nghttp2 library underneath. Both serve a connection's IDuplexPipe on the reactor that accepted it - h2c, or h2 over TLS - and multiplex all of that connection's streams on that one thread.

dotnet add package ioxide.http2

dotnet add package ioxide.nghttp2

Each depends on ioxide core alone. ioxide.nghttp2 also ships libioxide_nghttp2.so - nghttp2 1.70.90 statically linked behind a small C shim, needing only libc - built for linux-x64.

Two stacks, one shape#

Both offer the same pieces: a connection you hand a pipe, a request and a response as bytes, a writer for streamed responses and a reader for streamed request bodies. Only the names differ - Http2Connection, Http2Request, Http2Response, Http2ResponseWriter and Http2BodyReader against the same names with Nghttp2 in front - so a handler moves across by renaming types. Underneath:

Both lowercase response header names, send a status outside 100-599 as 500, and never let a parked handler hold up the others.

Which to use#

ioxide.http2 is the default: features land there first, it needs no native library, every limit is an option, and its streamed writer has real backpressure. ioxide.nghttp2 puts the reference implementation - continuously fuzzed, a decade of interop - under the same API, and is the safer of the two for large bodies returned whole, because nghttp2 sends them under flow control.

On speed, bench/README.md records the pure-C# server at 2.7x nghttp2's requests per second over h2c and 2.5x over TLS, at about a third of the CPU per request (August 2026: 2 reactors, 64 connections, 32 streams each, a 2-byte body). The gap narrowed to 1.48x at 16 reactors, a 2-byte body measures framing rather than data, and bench/h2-matrix.sh found nghttp2 ahead at 32 connections over 4 reactors - run it on your hardware before choosing on speed.

h2c and h2 over TLS#

h2c is HTTP/2 in cleartext with prior knowledge: the client opens with the HTTP/2 connection preface. The constructor that takes a TcpConnection wraps it in a TcpConnectionDualPipe for you:

reactor.TcpHandle = async (r, conn) =>
{
    try
    {
        await new Http2Connection(conn).RunBufferedAsync(request => Http2Response.Text("hello\n"));
    }
    finally
    {
        conn.DecRef();
    }
};

On nghttp2 the same line is await new Nghttp2Connection(conn).RunBufferedAsync(request => Nghttp2Response.Text("hello\n"));. There is no Upgrade: h2c: both servers write their SETTINGS first, and a client that opens with an HTTP/1.1 request instead of the preface gets those and a closed connection.

h2 over TLS is what browsers speak: they use HTTP/2 only over TLS, agreed by ALPN. Start a TlsService on each reactor with "h2" in TlsOptions.Alpn (the default is ["http/1.1"]), accept the handshake, and hand the connection a TlsConnectionDualPipe, which decrypts on the way in and encrypts on the way out. The HTTP/2 code is the h2c code.

• Alpn is ordered, server preference first. A client offering nothing on the list completes the handshake without ALPN, and TlsSession.NegotiatedAlpn is null. To serve h2 and HTTP/1.1 on one port, list both and branch on it, as Playground/Http2/Tls does.
• Plaintext the handshake already decrypted - for h2 usually the connection preface - is handed back by the pipe first.
• The pipe uses the kernel's halves when the handshake enabled kTLS, and any other IDuplexPipe works too: Playground/Http2/SslStream serves over SslStream.

A complete server#

h2 over TLS on port 8443, one reactor per core, with buffered dispatch: each handler runs once its request has fully arrived and returns the whole response.

using System.Text;
using ioxide;
using ioxide.http2;
using ioxide.tls;

// A PEM certificate chain (leaf first) and its private key.
string certPath = "server.crt";
string keyPath = "server.key";

var config = new ServerConfig
{
    ReactorCount = Environment.ProcessorCount,   // one reactor per core
    Udp = null,                                  // TCP only
    Tcp = new TcpOptions { Port = 8443 },
};

var tlsOptions = new TlsOptions
{
    CertificatePath = certPath,
    KeyPath = keyPath,
    Alpn = ["h2"],                               // the default is ["http/1.1"]
};

var threads = new Thread[config.ReactorCount];

for (int i = 0; i < threads.Length; i++)
{
    var reactor = new Reactor(i, config);

    // Each reactor terminates TLS with its own service, created on its own thread.
    reactor.OnStart = r => TlsService.Start(r, tlsOptions);

    reactor.TcpHandle = async (r, conn) =>
    {
        TlsSession? tls = null;
        try
        {
            tls = await r.GetService<TlsService>().AcceptAsync(conn);
            if (tls.NegotiatedAlpn != "h2")
            {
                return;   // the client did not offer h2
            }

            // Decrypts on the way in, encrypts on the way out: the HTTP/2 code is the h2c code.
            await using var pipe = new TlsConnectionDualPipe(conn, tls, ownsSession: false);
            await new Http2Connection(pipe).RunBufferedAsync(Handle);
        }
        catch (Exception e)
        {
            Console.Error.WriteLine($"connection failed: {e.Message}");
        }
        finally
        {
            tls?.Dispose();
            conn.DecRef();
        }
    };

    threads[i] = new Thread(reactor.Run) { Name = $"reactor-{i}" };
    threads[i].Start();
}

Console.WriteLine($"h2 on https://localhost:{config.Tcp!.Port}/");

foreach (Thread thread in threads)
{
    thread.Join();
}

// Runs once per request, on the reactor thread, after the request has fully arrived.
static Http2Response Handle(Http2Request request)
{
    ReadOnlySpan<byte> path = request.Path.Span;

    if (path.SequenceEqual("/"u8))
    {
        return Http2Response.Text("hello over h2\n");
    }

    if (path.SequenceEqual("/agent"u8))
    {
        return request.TryGetHeader("user-agent"u8, out ReadOnlyMemory<byte> agent)
            ? Http2Response.Text($"{Encoding.ASCII.GetString(agent.Span)}\n")
            : Http2Response.Text("no user-agent\n");
    }

    if (path.SequenceEqual("/upload"u8) && request.Method.Span.SequenceEqual("POST"u8))
    {
        // Buffered dispatch: the whole body is already here.
        return Http2Response.Text($"{request.Body.Length} bytes\n");
    }

    var notFound = new Http2Response { Status = 404, Body = "not found\n"u8.ToArray() };
    notFound.Headers.Add("content-type"u8.ToArray(), "text/plain"u8.ToArray());
    return notFound;
}

curl -k --http2 https://localhost:8443/agent

RunBufferedAsync returns when the client goes away or the connection fails, and disposes its own state; the pipe, the TLS session and the connection's reference are the caller's to release, as above. It also takes an asynchronous handler, Func<Http2Request, ValueTask<Http2Response>>.

Requests and responses#

Http2Request holds the request as it arrived, bytes throughout:

• StreamId, and the pseudo-headers Method, Path (with its query), Scheme and Authority - HTTP/2's Host, empty when the client sent none.
• Headers - the other fields in wire order, repeats kept. Field names are lowercase in HTTP/2, so look them up that way: TryGetHeader returns the first match. Cookies walks the pairs of every cookie field.
• Body - the whole body, or empty. BodyReader is null unless bodies are streamed.

That memory belongs to the connection and is recycled once the handler is done - when it returns, or when its task completes. Copy what you keep longer.

Http2Response has Status (200 by default), Headers and Body; Http2Response.Text(body, status) adds a UTF-8 content-type. It is copied into the connection's write path as soon as the handler finishes, so its memory may be reused afterwards, and it is sent as given: no content-length is added, and HEAD is not special-cased - answer HEAD with an empty body, or a strict client resets the stream.

Options#

Nghttp2Options has two of these, with the same defaults and meaning: MaxRequestBytes and StreamRequestBodies. The shim always advertises push off, 1000 concurrent streams and a 1 MiB stream window, and nghttp2 enforces the stream limit itself: a client that goes past it loses the connection to a GOAWAY. The rest are nghttp2's defaults.

Handlers that await#

All of a connection's streams are served by one loop on its reactor thread. Each pass parses every complete frame the last read delivered, dispatches every request that became ready, and ends in one flush carrying every response the pass produced. A handler that parks - on a database, an upstream, a timer - is detached: the loop moves on to the other streams and keeps reading, and when the awaited operation completes on the reactor, the handler resumes there and its response is written and flushed on its own. One slow request does not hold up the fast ones beside it.

reactor.OnStart = r =>
{
    TlsService.Start(r, new TlsOptions { CertificatePath = "server.crt", KeyPath = "server.key", Alpn = ["h2"] });
    HttpClientPool.Start(r, new HttpClientOptions { Host = "10.0.0.7", Port = 8080, PoolSize = 64 });
};

byte[] quotePath = "/quote"u8.ToArray();

reactor.TcpHandle = async (r, conn) =>
{
    HttpClientPool upstream = r.GetService<HttpClientPool>();
    TlsSession? tls = null;
    try
    {
        tls = await r.GetService<TlsService>().AcceptAsync(conn);
        await using var pipe = new TlsConnectionDualPipe(conn, tls, ownsSession: false);

        await new Http2Connection(pipe).RunBufferedAsync(async request =>
        {
            try
            {
                // Parks this stream only. The connection goes on reading frames and
                // answering its other streams until the upstream answers.
                using HttpClientResponse quote = await upstream.GetAsync(quotePath);

                // Copy: the upstream response's memory is recycled by Dispose, and the
                // h2 response is framed after this handler returns.
                return new Http2Response { Status = quote.Status, Body = quote.Body.ToArray() };
            }
            catch (HttpClientException)
            {
                return new Http2Response { Status = 502 };
            }
        });
    }
    finally
    {
        tls?.Dispose();
        conn.DecRef();
    }
};

One thread runs everything, so a handler needs no locks - and must never block it. Await, never .Result or .Wait(), and reach other services through clients on the reactor's own ring, created per reactor in OnStart: ioxide.httpclient, Postgres, Redis, timers. Every stream of every connection may be parked at once, so size such a pool for streams, not connections.

• A handler that fails. An asynchronous handler that throws is logged to standard error and its stream answered with a 500; the connection serves on. A synchronous handler that throws ends the whole connection, and every stream still in flight on it goes unanswered - catch inside it.
• A handler that takes long. The connection's read stays parked meanwhile, so TcpOptions.ReadTimeoutMs (60 s) closes it when nothing moves either way for that long; see lifetime and limits.
• Ending a connection. There is no graceful drain. Shutdown() only marks the connection broken: the loop ends after the pass in progress - or, while it waits, after the next read - and the connection closes without a GOAWAY. Streams still in flight are not answered.

Where to go next#