Run a local LLM, serve it to your whole fleet

You're running a 70B model on a single GPU box. Fifty downstream containers across your fleet need its output for the same query: they're scoring the same product catalog, generating embeddings, evaluating an experiment.

Naive answer: HTTP server with a queue, request batching, lock contention. Better answer for this shape: each query goes onto a pipe path with ?n=50. The model runs once. Fifty consumer containers curl ?n=50 and stream the same tokens at the same time, fanned out by the pipe. Slow ones get backpressure on their own connection.

You don't pay for fifty inferences. You don't run a request queue. You run the model. The pipe broadcasts.

This isn't every workload: it's the shape where N containers want the same answer. When that's your shape, the pipe is the cheapest fan-out you'll wire up.