Aller au contenu
#0850
Pipe

The dataset arrives with its provenance in the headers

You hand a client a 9GB parquet extract every Friday. The extract is fine. The questions come Monday: which source, which snapshot, how many rows, was it the version before or after the schema fix.

Now the answers travel with the bytes. The sender sets one header: X-Hoody-Pipe: source=warehouse-prod;snapshot=2026-08-28T02:00Z;rows=41230877;schema=v7. hoody-pipe forwards it to every receiver, alongside the Content-Disposition filename, and exposes it over CORS. The client's curl -v prints the provenance before the first byte lands on their disk. Their browser page can read the same header and show it next to the download button. No sidecar JSON to keep in sync, no README describing last week's file.

The bytes moved once and the metadata rode on top. The manifest is a response header.

The manifest is a response header.