コンテンツにスキップ
#0020
PipeContainers

A CI cache that's just two curl commands

CI caching is a tax. You pay S3 for storage you'll overwrite tomorrow, you pay egress every time a worker pulls, and you pay engineering time configuring the cache action's quirks.

Replace it with a pipe. The first job in your pipeline that produces node_modules.tar.zst streams it: tar c node_modules | zstd | curl -T -. The next twenty jobs that need it curl ... | zstd -d | tar x. With ?n=20 the producer waits until all twenty are connected, then pushes the cache through the pipe once, fanned out to all of them.

No S3 bucket. No lifecycle rules. No cache key collisions across branches: branches just use different paths. The pipe forgets the bytes the second the transfer ends, so you're not paying for storage you'll never read again.

The cache layer is HTTP. It always was. We just hadn't noticed.