Postgres and Redis

Redis

Redis in ioxide is StackExchange.Redis with its connections on the reactor's ring. ioxide.redis.stackexchange gives a multiplexer a transport whose connect, sends and receives are io_uring operations on a reactor, and StackExchange.Redis parses the replies and completes the commands on that reactor's thread.

<ItemGroup>
  <ProjectReference Include="../ioxide/src/clients/ioxide.redis.stackexchange/ioxide.redis.stackexchange.csproj" />
</ItemGroup>

It replaces ioxide.redis, ioxide's own RESP2 client, which is gone from the repository; 0.16.281 is its last version on NuGet.

A complete server#

Every reactor counts visits in Redis, with a multiplexer of its own:

using System.Text;
using ioxide;
using ioxide.redis.stackexchange;
using ioxide.utils;
using StackExchange.Redis;

var config = new ServerConfig
{
    ReactorCount = Environment.ProcessorCount,
    Tcp = new TcpOptions { Port = 8080 },
};

var threads = new Thread[config.ReactorCount];
for (int i = 0; i < threads.Length; i++)
{
    var reactor = new Reactor(i, config);

    // A multiplexer per reactor, its connections on this reactor's ring. Keep the task: the
    // connect completes on the loop that starts once OnStart returns.
    reactor.OnStart = r => r.AddService(IoxideRedis.ConnectAsync(r, "127.0.0.1:6379"));

    reactor.TcpHandle = async (r, conn) =>
    {
        IDatabase db = (await r.GetService<Task<ConnectionMultiplexer>>()).GetDatabase();
        try
        {
            while (true)
            {
                RecvSnapshot snapshot = await conn.ReadAsync();
                bool request = false;
                while (conn.TryGetItem(snapshot, out SpscRecvRing.Item item))
                {
                    if (item.HasBuffer)
                    {
                        request = true;               // a real server parses the request here
                        conn.ReturnBuffer(in item);
                    }
                }

                if (request)
                {
                    // Sent on this reactor's ring, completed on this reactor's thread.
                    long visits = await db.StringIncrementAsync("visits");
                    string body = $"visit {visits}";
                    conn.Write(Encoding.ASCII.GetBytes($"HTTP/1.1 200 OK\r\nContent-Length: {body.Length}\r\n\r\n{body}"));
                    await conn.FlushAsync();
                }

                if (snapshot.IsClosed) return;
                conn.ResetRead();
            }
        }
        finally
        {
            conn.DecRef();
        }
    };

    threads[i] = new Thread(reactor.Run) { Name = $"reactor-{i}" };
    threads[i].Start();
}

foreach (Thread thread in threads) thread.Join();

Run it beside a Redis (docker run --rm -p 6379:6379 redis:7-alpine), and the requests answer visit 1, visit 2, ... The repository's Playground/Clients/RedisStackExchange is a fuller server - a GET route, INCR and a cache-aside route - with a knob, PLAYGROUND_REDIS_TRANSPORT=sockets, that runs the same handler on StackExchange.Redis's own sockets.

How it fits together#

• IoxideTunnel(reactor) is a StackExchange.Redis Tunnel: each connection of a multiplexer configured with it is a RingSocket on that reactor, and StackExchange.Redis opens no socket of its own.
• IoxideRedis.ConnectAsync(reactor, options) sets ConfigurationOptions.Tunnel to an IoxideTunnel on that reactor - replacing any tunnel already set - and connects. Call it from OnStart and keep the task as a service: never wait on it there, since the connect completes on the loop that starts after OnStart returns.
• One multiplexer per reactor. Then a handler's commands are staged, sent, parsed and completed on its own thread, and the await resumes there. A multiplexer shared by every reactor also works - the transport takes writes and flushes from any thread - but each command from another reactor then crosses threads on the way out and back.
• Async APIs only. On the reactor that owns the multiplexer, a synchronous call - StringGet rather than StringGetAsync - blocks the thread that would send the command and read its reply. The reactor stalls for StackExchange.Redis's SyncTimeout, 5 seconds by default, and the call throws RedisTimeoutException.

What runs where#

Concurrent commands share a connection, as StackExchange.Redis always does:

// All three are in flight at once, on one connection.
Task<long>[] increments = [db.StringIncrementAsync("a"), db.StringIncrementAsync("b"), db.StringIncrementAsync("c")];
long[] values = await Task.WhenAll(increments);

Options#

Everything else is StackExchange.Redis's own ConfigurationOptions - endpoints, password, database, timeouts - and applies as usual. To size the receive buffer, give the tunnel yourself:

var options = new ConfigurationOptions
{
    EndPoints = { { "redis.internal", 6379 } },   // a name is resolved off the reactor
    Password  = Environment.GetEnvironmentVariable("REDIS_PASSWORD"),
    Tunnel    = new IoxideTunnel(reactor, receiveBufferSize: 256 * 1024),
};
Task<ConnectionMultiplexer> redis = ConnectionMultiplexer.ConnectAsync(options);

StackExchange.Redis opens an interactive and a subscription connection to each server, so N reactors with a multiplexer each hold 2N connections to every Redis server - 64 for the server above on 32 reactors.

The whole API#

Every command goes through the same transport, so StackExchange.Redis's whole API is there, and its documentation applies. The Redis suite runs commands, 1000 concurrent INCRs, a 1 MiB value that arrives across several receives, pub/sub, and a connection the server kills that comes back. Pub/sub goes over the subscription connection, on the same ring:

ISubscriber subscriber = mux.GetSubscriber();
RedisChannel events = RedisChannel.Literal("events");
await subscriber.SubscribeAsync(events, (channel, message) => Console.WriteLine($"{channel}: {message}"));
await subscriber.PublishAsync(events, "hello");

Measured#

From bench/redis-matrix.sh: Playground/Clients/RedisStackExchange answers GET / with one Redis GET, and its PLAYGROUND_REDIS_TRANSPORT knob is the only difference between the two arms. Both arms run pinned to the same 2 CPUs with 2 reactors, Redis 7 has a core of its own on the host network, and wrk -t4 -c64 -d10s runs on the remaining performance cores. The server's CPU per request comes from /proc, over three interleaved passes on one machine over loopback.

Over ioxide, StackExchange.Redis serves 1.59x the requests per second at 0.73x the CPU per request, best of three. The ioxide arm kept its two CPUs 97-99% busy; on its own sockets, StackExchange.Redis used 85% of the same two and went no further. BENCH.md has the machine, the placement and how to run it.

Limits#

• An experimental seam. StackExchange.Redis promises nothing about it between versions; the package is built against 3.4.0 and pins it.
• No TLS through the tunnel yet. A configuration that asks for TLS is refused when the connection is made: the connect fails with RedisConnectionException, and its log gives the reason - the ioxide transport does not do TLS yet: connect without TLS, or leave Tunnel unset for StackExchange.Redis's own sockets.
• TCP only. A Unix domain socket endpoint is refused: the ioxide transport connects over TCP only, not to a UnixDomainSocketEndPoint.
• A close is reported quietly on 3.4.0. When the server ends a connection, StackExchange.Redis 3.4.0, the version the package pins, raises no ConnectionFailed, and a command caught by the close fails with TaskCanceledException rather than RedisConnectionException. The connection still comes back. StackExchange.Redis reported a failure only for a connection with a Stream, and a transport has none. 3.4.5 fixes it, and the package still pins 3.4.0.

Where to go next#