Aller au contenu
#0017
PipeAgent

Stream LLM tokens to anything that reads HTTP

You're building a multi-step agent. Step 3 generates a long response. Step 4 needs to start consuming the tokens before step 3 is done: otherwise the user waits four seconds staring at nothing.

The usual answer: SSE endpoint, queue, Redis pub/sub, framework. The pipe answer: step 3 streams its tokens with curl -T - to a path. Step 4 curls the same path. Tokens move from generator to consumer at line speed, no buffering, no broker.

The same path works for everything downstream: a frontend EventSource, a logging container, an evaluator agent that decides whether to interrupt. Up to 256 listeners on the same stream, fanned out by the pipe with backpressure handled per-receiver. No SDK, no client library, no setup. The endpoint exists because two curls touched it.

The LLM streams. The pipe streams. The reader streams. No middle layer.