Skip to content
#0012
Pipe

Push one build to thirty CI workers at once

Your matrix CI fans out across thirty test runners. Each one needs the same 800MB Docker image you just built. The naive flow: push to a registry, thirty workers pull from the registry, thirty network round-trips, thirty cold caches, thirty bandwidth bills.

The pipe flow: the build container streams the tarball once to a path with ?n=30. All thirty test workers curl ?n=30 the same URL. The transfer doesn't start until everyone's connected, then the bytes go through the pipe one time. The server holds nothing: it's a fan-out router with no disk.

A 30-way push that took ninety seconds and an S3 hit takes twelve seconds and a single egress. The slowest receiver sets the pace, but no one re-downloads, and no registry credentials get rotated.

One sender. Thirty receivers. Zero S3 bills.