Core runtime

Core runtime

The engine under every ioxide server and client. A reactor is one io_uring ring and the one thread that drives it: a server runs one per core, each owns its listeners, connections, clients and timers outright, and every completion resumes the code waiting for it inline, on that thread.

dotnet add package ioxide

Linux 6.1 or later, .NET 10 or 11. The ring is driven through raw system calls into libc - no liburing. The TLS in the same package brings its own BoringSSL, libioxide_boringssl.so for linux-x64, and loads it only when it is used.

What a reactor is#

One reactor is one thread, one io_uring ring, an SO_REUSEPORT listener for each TCP port, and the table of the connections it accepted. Its thread is the only one that writes to the ring, to the ring of receive buffers and to the connection table, so none of them needs a lock; other threads reach them through queues the reactor drains.

Every reactor binds the same ports, and the kernel spreads new connections across their listeners. A connection is read, handled, answered and closed on the reactor that accepted it; nothing moves it to another thread. The clients work the same way: an HTTP or Postgres pool, or a Redis multiplexer, opened on a reactor submits its connects, sends and receives to that reactor's ring, so a handler that calls an upstream never leaves its thread on the way out or back.

A handler is ordinary async C#. An await on the ring parks it until the kernel posts the completion; the reactor reads the completion and calls straight back into the handler, inside its loop - no thread pool, no queue in between. Anything else a handler awaits, Task.Run for one, is brought back to the reactor by the reactor's synchronization context, at a price.

The core moves bytes and stays out of HTTP. TLS, HTTP/2 and HTTP/3 ride on top of it, and a server that speaks HTTP/1.1 frames it itself, as the one below does.

The smallest server#

An HTTP responder on port 8080 with a reactor per core: Playground/Tcp/Raw without its knobs.

using ioxide;
using ioxide.utils;

var config = new ServerConfig
{
    ReactorCount = Environment.ProcessorCount,   // one reactor per core
    Tcp = new TcpOptions { Port = 8080 },        // every reactor listens on 8080
};

byte[] response = "HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\nok"u8.ToArray();
var threads = new Thread[config.ReactorCount];

for (int i = 0; i < threads.Length; i++)
{
    var reactor = new Reactor(i, config);

    // Runs once per accepted connection, on this reactor's thread.
    reactor.TcpHandle = async (r, conn) =>
    {
        try
        {
            while (true)
            {
                // A receive on the ring. The handler resumes inline when it completes.
                RecvSnapshot snapshot = await conn.ReadAsync();

                // The bytes stay in the reactor's receive buffers until handed back.
                // A real server parses its requests here; this one only answers.
                while (conn.TryGetItem(snapshot, out SpscRecvRing.Item item))
                {
                    if (item.HasBuffer) conn.ReturnBuffer(in item);
                }

                conn.Write(response);       // into the connection's write buffer
                await conn.FlushAsync();    // one send on the ring

                if (snapshot.IsClosed) return;
                conn.ResetRead();
            }
        }
        finally
        {
            conn.DecRef();   // the handler's hold on the connection, released once
        }
    };

    threads[i] = new Thread(reactor.Run) { Name = $"reactor-{i}" };
    threads[i].Start();
}

foreach (Thread thread in threads)
{
    thread.Join();
}

curl http://127.0.0.1:8080/

• The config is a record set with init. Tcp sets the listener's port, and everything else keeps its default (ServerConfig).
• The handler runs once per accepted connection, on the reactor that accepted it, and loops: wait for bytes, hand the receive buffers back, write the response, flush it. Reading and writing covers the connection's API, and Lifetime and limits the DecRef that ends it.
• A thread per reactor runs reactor.Run, which creates the ring on that thread, opens the listener and runs the loop until the reactor stops.

It answers each read with one response, which suits curl and wrk. A real server parses the bytes it reads and answers each request in them.

Handlers#

All three run on the reactor thread and are set before Run. A TCP or QUIC handler that throws is logged and its connection released; a datagram handler that throws is logged and the datagram dropped. TCP is on by default, so a reactor without a TcpHandle fails every connection it accepts: give it Tcp = null to accept none.

Starting and stopping#

new Reactor(id, config) only records its arguments. Run does the rest, on the thread that calls it: it installs the reactor's synchronization context, creates the ring, opens the TCP listeners and UDP sockets, registers the receive buffers, runs OnStart, arms the accepts and enters the loop. It returns once the reactor has stopped and torn down. A server that stops on Ctrl+C, its handler in a method Handle:

var reactors = new Reactor[config.ReactorCount];
var threads = new Thread[config.ReactorCount];

for (int i = 0; i < reactors.Length; i++)
{
    reactors[i] = new Reactor(i, config)   // ids 0 .. ReactorCount - 1, one config object
    {
        TcpHandle = Handle,
        // A fault ends this reactor only. Without a handler it ends the process.
        OnFault = (r, e) => Console.Error.WriteLine($"reactor {r.ShardIndex} stopped: {e.Message}"),
    };
    threads[i] = new Thread(reactors[i].Run) { Name = $"reactor-{i}" };
    threads[i].Start();
}

Console.CancelKeyPress += (_, e) =>
{
    e.Cancel = true;
    foreach (Reactor reactor in reactors) reactor.Stop();   // safe from any thread
};

foreach (Thread thread in threads) thread.Join();   // Run returns after its teardown

• Ids run from 0 to ReactorCount - 1, and every reactor gets the same ServerConfig object. Nothing creates reactors for you: ReactorCount says how many you will. QUIC relies on both - it stamps a reactor's id into the connection IDs it mints, and finds the other reactors through the shared config to forward a datagram to its owner.
• Stop sets a flag and wakes the loop, from any thread. The loop finishes its turn and exits, and Run closes the listeners, the UDP sockets and every connection the reactor accepted, then the ring - on the reactor thread, as the ring requires. Outbound sockets belong to the clients that opened them. Nothing is drained first: a graceful close - an HTTP/3 GOAWAY from Nghttp3Connection.Shutdown, say - is the protocol's job, done before Stop.
• OnFault is raised on the reactor thread when the reactor ends because of an exception rather than Stop: a kernel that refuses the ring, a throw from OnStart, an io_uring_enter error the loop cannot carry on from. It runs before the teardown. Without a handler the exception leaves Run, which on a plain thread ends the process; with one, the other reactors keep serving.

Per-reactor state: OnStart, services and tickers#

What belongs to a reactor - a client pool, a cache, a counter - is created on it, in OnStart. That runs on the reactor thread once the ring, the listeners and the buffers exist and before the loop starts, so a client opened there binds to this reactor's ring, and what it submits completes once the loop is running. A handler finds it again by type - here the quick start's, counting its responses:

reactor.OnStart = r =>
{
    var stats = new Stats();
    r.AddService(stats);                  // keyed by type: r.GetService<Stats>()
    r.AddTicker(() => stats.OnTick(r));   // every 250 ms, on this reactor's thread
};

reactor.TcpHandle = async (r, conn) =>
{
    Stats stats = r.GetService<Stats>();
    try
    {
        while (true)
        {
            RecvSnapshot snapshot = await conn.ReadAsync();
            while (conn.TryGetItem(snapshot, out SpscRecvRing.Item item))
            {
                if (item.HasBuffer) conn.ReturnBuffer(in item);
            }

            conn.Write(response);
            await conn.FlushAsync();
            stats.Responses++;            // no lock: only this thread touches it

            if (snapshot.IsClosed) return;
            conn.ResetRead();
        }
    }
    finally
    {
        conn.DecRef();
    }
};

sealed class Stats
{
    public long Responses;
    private int _ticks;

    public void OnTick(Reactor r)
    {
        if (++_ticks % 4 == 0)            // four ticks: about a second
        {
            Console.WriteLine($"[r{r.ShardIndex}] {Responses} responses");
        }
    }
}

• AddService and GetService keep one object per type, keyed by the type argument. A second AddService of a type replaces the first, and GetService of a type never added throws InvalidOperationException. The ioxide clients register themselves this way - HttpClientPool.Start, PgPool.Start, TlsService.Start - so with two pools of one type on a reactor, GetService finds only the second: keep the references Start returns.
• AddTicker registers a callback the loop runs on the reactor thread every Reactor.TickMs, 250 ms, the first a tick after the loop starts. A ticker runs for the reactor's life - there is no removal - and one that throws is logged and called again on the next tick. The client pools sweep their connections this way, and the reactor sweeps its own TCP timeouts (tickers and timeouts).
• Both are plain collections owned by the reactor thread: call them from OnStart or a handler, never from another thread.

Rules that keep it fast and correct#

• Touch a reactor only from its thread. Its connections, clients and services are single-threaded by design. From another thread, hand work over with ScheduleOnReactor (work from other threads); OnReactorThread says where you are.
• Await; never block. Blocking the reactor thread with .Result, .Wait() or GetAwaiter().GetResult() on anything that needs the reactor to finish - a ring operation, an async method that resumes on it - waits for a completion only that thread can deliver: the reactor deadlocks, with every connection on it.
• No blocking calls. A synchronous DNS lookup, file read or Thread.Sleep in a handler stops every connection on the reactor for as long as it takes. The ring has a form of each that does not block: sockets, files, timers.
• Leaving the reactor costs. Task.Run and Task.Delay come back to the reactor through the thread pool and a cross-thread wake; on the repository's benchmark a Task.Run per request doubled the CPU each request cost (priced here). After ConfigureAwait(false) the code does not come back at all.
• Do not loop on Task.Yield(). On a reactor it reaches neither the thread pool nor the kernel: the continuation goes to the reactor's own queue, which is drained until it is empty, so a loop of yields keeps every other connection on the reactor waiting until it ends.

Where to go next#