Files and timers

Timers

ioxide.timer's RingTimer waits on the reactor's own ring. A wait is one IORING_OP_TIMEOUT: the kernel holds the deadline, the completion resumes the waiting code inline on the reactor, and nothing is allocated per wait.

dotnet add package ioxide.timer

It depends on ioxide core alone. The operation underneath, IRingHost.SubmitTimeout, is core's; RingTimer gives it the shape of the other ring clients, as RingFile does for reads.

A handler that answers later#

A server on port 8080 that answers GET /250 after 250 ms:

using System.Buffers.Text;
using System.Text.Unicode;
using ioxide;
using ioxide.timer;
using ioxide.utils;

var config = new ServerConfig
{
    ReactorCount = Environment.ProcessorCount,
    Tcp = new TcpOptions { Port = 8080 },
};

var threads = new Thread[config.ReactorCount];
for (int i = 0; i < threads.Length; i++)
{
    var reactor = new Reactor(i, config);

    reactor.TcpHandle = async (r, conn) =>
    {
        // One timer for the connection's life: it carries one wait at a time, re-armed per request.
        var timer = new RingTimer(r);
        try
        {
            while (true)
            {
                RecvSnapshot snapshot = await conn.ReadAsync();

                int ms = -1;
                while (conn.TryGetItem(snapshot, out SpscRecvRing.Item item))
                {
                    if (item.HasBuffer)
                    {
                        if (ms < 0) ms = ParseDelay(item.AsSpan());
                        conn.ReturnBuffer(in item);
                    }
                }

                if (ms >= 0)
                {
                    // One IORING_OP_TIMEOUT: the kernel holds the deadline, and the completion
                    // resumes this handler inline, on this reactor.
                    int result = await timer.DelayAsync(ms);
                    Answer(conn, ms, result);
                    await conn.FlushAsync();
                }

                if (snapshot.IsClosed) return;
                conn.ResetRead();
            }
        }
        finally
        {
            conn.DecRef();
        }
    };

    threads[i] = new Thread(reactor.Run) { Name = $"reactor-{i}" };
    threads[i].Start();
}

foreach (Thread thread in threads) thread.Join();

// "GET /250 HTTP/1.1" -> 250, at most 10 s: the client picks the wait, so it is capped.
static int ParseDelay(ReadOnlySpan<byte> request)
{
    int slash = request.IndexOf((byte)'/');
    return slash >= 0 && Utf8Parser.TryParse(request[(slash + 1)..], out int ms, out _) ? Math.Clamp(ms, 0, 10_000) : 0;
}

// Expiry is -ETIME, which RingTimer.Expired reads as success; anything else is an errno.
static void Answer(TcpConnection conn, int ms, int result)
{
    Span<byte> body = stackalloc byte[32];
    int bodyLength;
    if (RingTimer.Expired(result)) Utf8.TryWrite(body, $"waited {ms} ms", out bodyLength);
    else Utf8.TryWrite(body, $"errno {result}", out bodyLength);

    Span<byte> slab = conn.GetSpan(128);
    Utf8.TryWrite(slab, $"HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: {bodyLength}\r\n\r\n", out int headLength);
    body[..bodyLength].CopyTo(slab[headLength..]);
    conn.Advance(headLength + bodyLength);
}

Try it with curl http://127.0.0.1:8080/250. The timer is made once per connection and re-armed for every request, and since the client picks the delay, it is capped. The repository's Playground/Clients/Timer is this server, with a knob that swaps Task.Delay in.

RingTimer#

sealed class RingTimer, in namespace ioxide.timer.

The result is the ring's, not translated: a wait that ran its course completes with -ETIME, which for a timeout is success, and any other negative value is an errno. Test it with Expired, never with result >= 0. A zero delay still goes through the ring as a 1 ns timeout, and completes with -ETIME as well.

Rules#

• One wait at a time. A second DelayAsync while one is in flight throws InvalidOperationException: "An operation is already in flight on this RingOpSource." Hold a timer per connection, or per thing that waits; deadlines that overlap need a timer each. Await each result once, as with any ValueTask.
• No cancel. Neither RingTimer nor IRingHost can withdraw a wait: it runs its course, and only then is the timer free again. A RingTimer paces and delays; when it fires, it cannot abandon another ring operation still waiting.
• On its own reactor. Wait on the reactor the timer was made for, from OnStart on, and await it there. Called from another thread, DelayAsync is queued to the reactor, which is woken to submit it - and the completion still arrives on the reactor's thread, so the code after the await runs there.
• A wait does not outlive its reactor. Nothing completes a wait that is in flight when the reactor stops: code awaiting it never resumes.
• The read timeout does not run during a wait. TcpOptions.ReadTimeoutMs counts only while a read is parked, so a handler waiting on a timer between reads is not cut off by it.

How a wait runs#

1. DelayAsync arms the timer's RingOpSource and calls IRingHost.SubmitTimeout.
2. The reactor takes a slot in its table of operations in flight, and writes the deadline into that slot's timespec, in a native block that runs parallel to the table: the deadline lives exactly as long as the operation, and nothing is allocated. Then it fills an SQE: IORING_OP_TIMEOUT, pointing at that one timespec, with a completion count of zero - so the timeout waits on time alone, not on a number of other completions.
3. The SQE needs no system call of its own: it leaves with the reactor's next io_uring_enter, along with everything else submitted that turn.
4. When the deadline passes, the kernel posts a completion with -ETIME. It wakes the reactor like any other; the reactor frees the slot and completes the awaitable, which runs the waiting code inline.

The table starts at ServerConfig.OpSlots (4096) slots, shared with every other ring operation of the reactor's clients, and doubles when it runs out; the repository's tests check that a wait queued just before it grows keeps its deadline. The reason for all this: a benchmark that answers after a delay named in the request holds 32,000 live deadlines at 32,000 connections, and a timerfd for each would cost a descriptor and a system call per wait.

RingTimer or Task.Delay#

await Task.Delay(ms) in a handler works: the reactor's synchronization context brings the continuation back to the reactor (how). It takes the long way round:

The repository has no measurement of the difference yet. The Clients/Timer sample's useTaskDelay knob - PLAYGROUND_TASK_DELAY=1 - runs the same server on Task.Delay, so the two can be priced against each other, but no file in bench/results records either. The benchmark suite registers the sample at PLAYGROUND_DELAY_MS=0, which prices RingTimer's shortest wait and not Task.Delay, whose zero delay waits for nothing: compare the two at the same delay above zero. What leaving the reactor costs in general is measured in How it works.

How the rest of ioxide keeps time#

No part of ioxide itself uses RingTimer. Its timeouts ride the reactor's tick: each reactor runs its tickers every 250 ms (Reactor.TickMs), and no timer drives them - the loop's wait for completions is bounded by the next tick, and a busy reactor checks the clock after every batch (Tickers and timeouts). The tick carries:

• the TCP sweep: ReadTimeoutMs and SendTimeoutMs, 60 000 by default (Timeouts), FINs left behind a flush, and accepts paused for want of descriptors;
• TlsService's handshake timeouts, and the QUIC idle sweep;
• the Postgres pool's CommandTimeoutMs, 30 000 by default, and its reconnects; the HTTP client pools' sweeps.

So each of those fires at the first tick after its deadline, never before it. One deadline does not wait for the tick: opening a Postgres connection, bounded by the same CommandTimeoutMs, submits a timeout of its own to the ring (IRingHost.SubmitTimeout, which RingTimer uses too). QUIC's loss and retransmission timers are finer: the earliest one bounds the loop's wait as well. They all read Native.MonotonicMs, a coarse monotonic clock in milliseconds (reference).

For periodic work of your own, choose by precision:

reactor.OnStart = r =>
{
    _ = EveryAsync(r, 10, flushMetrics);   // every 10 ms, on this reactor's thread
    r.AddTicker(sweep);                    // or every tick, about 250 ms, with no timer at all
};

static async Task EveryAsync(Reactor reactor, int ms, Action work)
{
    var timer = new RingTimer(reactor);
    while (true)
    {
        await timer.DelayAsync(ms);   // the next wait is armed after the work, so the period drifts by it
        work();
    }
}

AddTicker, from OnStart, runs a callback on the reactor's thread every tick. There is no way to remove one; an exception from it is logged as [r0] ticker faulted: ... and the ticks go on; a tick that comes late is not made up. A RingTimer loop keeps its own interval, which drifts by the time the work takes, because each wait is armed after it.

Where to go next#