HTTP/2

Streaming and flow control

A response written as it is produced, a request body read as it arrives, and the two flow-control windows that pace both - how each works on ioxide.http2 and on ioxide.nghttp2.

Two switches#

The two directions are separate choices, because they solve different problems. RunAsync instead of RunBufferedAsync streams the response: the handler gets a writer, which lets a body exist with no known length - or no end at all. StreamRequestBodies = true streams the request: the handler runs as soon as the headers are in and reads the body as it arrives, so an upload is held one window at a time instead of whole. Use either alone, or both: Playground/Http2/ManagedStreamedBoth echoes an upload chunk by chunk, holding neither direction whole.

Streamed responses#

return new Http2Connection(conn).RunAsync(async (request, writer) =>
{
    var head = new Http2Response { Status = 200 };
    head.Headers.Add("content-type"u8.ToArray(), "text/csv"u8.ToArray());
    writer.WriteHeaders(head);   // once, before any body byte

    for (int row = 0; row < 1_000_000; row++)
    {
        Span<byte> line = writer.GetSpan(32);
        Utf8.TryWrite(line, $"{row},{row % 97}\n", out int written);
        writer.Advance(written);

        if (row % 1000 == 999)
        {
            // Out as DATA frames, waiting while the stream's or the connection's window is shut.
            FlushResult flushed = await writer.FlushAsync();
            if (flushed.IsCompleted)
            {
                return;   // the client reset the stream, or the connection is gone
            }
        }
    }

    // Returning sends what is still staged, then END_STREAM.
});

The writer, Http2ResponseWriter, is an IBufferWriter<byte>, so a serialiser or a framework's response sink writes into it directly.

• Headers first, once. WriteHeaders takes an Http2Response whose Body is empty (anything else throws). No content-length is added: END_STREAM marks the end.
• Staged until flushed. GetSpan and Advance fill a pooled staging buffer, which holds everything written since the last flush - flush as you go to keep it small.
• The flush waits for credit. FlushAsync frames the staged bytes as DATA, each frame bounded by the stream's window, the connection's window and the client's maximum frame size. With no credit left it first sends the frames already made - a client returns credit only for bytes it has received - then waits for a WINDOW_UPDATE on the stream or the connection, or a SETTINGS that raises the window. That wait is the backpressure: a client that stops reading stops the producer.
• Gone is reported, not thrown. Once the client resets the stream or the connection ends, FlushAsync returns a result with IsCompleted set, and nothing written afterwards reaches anyone.
• The end. When the handler returns, the writer is completed for it: the rest of the staging, then END_STREAM. CompleteAsync() does that earlier, if the handler has more to do. A handler that never wrote headers gets a 500. A handler that throws gets a 500 if no headers went out yet, and an RST_STREAM with INTERNAL_ERROR otherwise - an END_STREAM would pass a truncated body off as whole.

Flushes made while the connection is still in the read pass that dispatched the handler do not each cost a write: they ride the pass's one flush with every other response, until this response has staged 16 KiB since its last real write. Once the pass's flush has gone by - typically after the handler has awaited something - a flush is a write of its own. The writer is pooled per connection, so do not keep it once the handler has returned.

On nghttp2#

Nghttp2Connection.RunAsync and Nghttp2ResponseWriter have the same shape, but nghttp2 owns the framing and pulls body bytes when it is ready to send. FlushAsync copies the staged bytes into nghttp2's buffer for the stream, wakes the stream, and writes whatever nghttp2 produces now: the bytes are handed over, not necessarily on the wire. Smaller differences: a flush before WriteHeaders, and a handler that writes no headers at all, send a 200 where ioxide.http2's writer throws or answers 500, and CompleteAsync is not public.

Streamed request bodies#

const long Limit = 64L * 1024 * 1024;
var options = new Http2Options { StreamRequestBodies = true };

return new Http2Connection(conn, options).RunBufferedAsync(async request =>
{
    Http2BodyReader body = request.BodyReader!;   // set whenever StreamRequestBodies is on
    long total = 0;

    while (true)
    {
        // Handing a chunk over is what returns its credit to the client.
        ReadOnlyMemory<byte> chunk = await body.ReadAsync();
        if (chunk.IsEmpty)
        {
            break;   // the end of the body
        }

        total += chunk.Length;
        if (total > Limit)
        {
            // MaxRequestBytes does not apply to a streamed body; the handler sets its own.
            return Http2Response.Text("too large\n", 413);
        }

        // Use or copy chunk here: the next ReadAsync recycles its memory.
    }

    return Http2Response.Text($"{total} bytes\n");
});

• Dispatched at the headers. The handler runs once the header block is complete, with BodyReader set and Body empty. A request that ended with its headers reads empty at once.
• Chunks as they arrived. On ioxide.http2 a chunk is one DATA frame's payload, padding removed. Its memory is valid until the next ReadAsync or the end of the handler. One reader, on the reactor thread.
• An empty chunk ends the body - after END_STREAM, but also when the client resets the stream or the connection drops. A handler cannot tell those apart from the chunk alone; when completeness matters, compare the bytes read against the content-length the client sent.
• No size limit applies. MaxRequestBytes bounds a body held whole, and a streamed body is never held whole, so the limit is the handler's, as above.
• Answering early. A handler may answer without reading to the end. Chunks still queued are dropped and their credit returned, and DATA that arrives later is credited and discarded on arrival. The stream is not reset, so the client may go on sending.

On nghttp2 it is the same with Nghttp2Options and Nghttp2BodyReader:

var options = new Nghttp2Options { StreamRequestBodies = true };

return new Nghttp2Connection(conn, options).RunBufferedAsync(async request =>
{
    long total = 0;
    ReadOnlyMemory<byte> chunk;
    while (!(chunk = await request.BodyReader!.ReadAsync()).IsEmpty)
    {
        total += chunk.Length;
    }

    return Nghttp2Response.Text($"{total} bytes\n");
});

Its session runs with nghttp2's automatic WINDOW_UPDATE off, and every byte nghttp2 delivers is consumed exactly once: when the handler reads it, when the handler returns leaving it unread, when its stream is reset, or on arrival if nobody will read it.

Flow control#

HTTP/2 paces every DATA frame twice: by its stream's window, and by the connection's, which all the streams share. Each side opens with the windows the other granted, and grants more with WINDOW_UPDATE as it consumes.

Request bodies#

• Buffered: credit goes back as DATA arrives - on ioxide.http2 a WINDOW_UPDATE for the stream and one for the connection per frame, riding the pass's flush; on nghttp2, nghttp2's own window updates. Memory is bounded by MaxRequestBytes instead.
• Streamed: credit goes back only as the handler reads. Padding is credited on arrival, since no handler sees it.
• A slow reader stops replenishing, so the client runs out of window and stops sending: memory stays bounded by the window. Because the connection window is 65,535 bytes, a handler holding that much unread shuts it for the whole connection - no stream on it can send body bytes until that handler reads, or returns and its unread bytes are credited. Requests without bodies still get through; HEADERS are not flow-controlled. It also means an InitialWindowSize above 65,535 does not let a client run further ahead.

Responses#

• ioxide.http2, streamed: the writer waits for credit, as above.
• ioxide.http2, buffered: DATA goes out until the client's connection window is spent; if the body is larger, the rest is dropped and the stream reset with FLOW_CONTROL_ERROR. The client's stream window is not consulted on this path - a known gap, kept as a pending test in Http2FlowControlTests. The protocol starts a client at 65,535 bytes and clients commonly raise it when they connect, so how much fits depends on the client: send any body that may not fit through the writer.
• nghttp2: nghttp2 honours both windows itself. A buffered body waits in its copy until credit arrives; DATA frames are at most 16 KiB whatever the client allows.

A client that holds a window shut and sends nothing is eventually cut by the connection's clocks, ReadTimeoutMs and SendTimeoutMs (60 s each); see lifetime and limits.

Trailers#

• Buffered request: the trailer section's fields are appended to Headers, after the header fields, and the request is dispatched once, with its body. Both stacks.
• Streamed request: the handler is already running, so the trailers are decoded - HPACK's table is connection state and must stay in step - and dropped. Their END_STREAM still ends the body. They never reach the running request, nor become a request of their own.
• Responses carry no trailers: neither stack has an API for them.

What the server resets#