Skip to content
use-cases / move-200gb-between-clouds-with-two-curls / hero
PIPE · CROSS-CLOUD MIGRATION

Move 200GB between clouds with two curls

A 200GB Postgres backup in Frankfurt. A fresh box in Singapore. You skip the S3 round-trip. One curl pipes pg_dump to a Hoody pipe URL; one curl on the other side streams it straight into psql. Bytes are in flight, never at rest.

Read the pipe docs
use-cases / move-200gb-between-clouds-with-two-curls / mechanism

It's not a bucket. It's a router.

Hoody Pipe is a named path on an HTTP server. The sender PUTs a stream to it, the receiver GETs the same path, and the server splices the two together. Nothing is written to disk; the pipe holds zero bytes by design.

Bytes between two curls5-MIN TTL · NO STORAGE LAYER
01 · SENDER

PUT to the path

From the source box, pg_dump | gzip | curl -T - to the pipe URL. The PUT body streams as fast as TCP backpressure allows. The server holds the connection until a receiver shows up on the same path.

PUT /api/v1/pipe/migration
02 · ROUTER

Splice in memory

When the receiver's GET lands on the same path, Hoody splices the upload's bytes directly into the download's response. No buffer, no on-disk staging, no async commit — just a direct stream between two HTTP sockets.

0 bytes on disk
03 · RECEIVER

GET it as a stream

From the destination box, curl GETs the path and pipes the response through gunzip | psql. The receiver-side stream finishes the second the sender's last byte lands. No retry, no manifest, no cleanup.

GET /api/v1/pipe/migration

Connection ordering doesn't matter — the receiver can curl first and block until the sender connects (or vice-versa), up to the 5-minute pipe TTL. Backpressure flows end-to-end: a slow psql throttles the curl on the source. There's no queue to overflow because there's no queue.

use-cases / move-200gb-between-clouds-with-two-curls / commands

The actual two commands

These are not pseudocode. Open two terminals on the two servers, run one each, and watch a 200GB backup leave one cloud and land in another.

frankfurt:~ · sender
PUT · SENDERFrankfurt → pipe
# 1. Stream the live database to the pipe$pg_dump --format=custom mydb | gzip \ | curl -T - https://pipe.containers.hoody.com/api/v1/pipe/migration# 2. The server replies with status messages[INFO] Waiting for 1 receiver(s) to connect...[INFO] Streaming to 1 receiver(s)...[INFO] Upload complete.[INFO] Transfer complete.
singapore:~ · receiver
GET · RECEIVERpipe → Singapore
# 1. Pipe straight into the new database$curl https://pipe.containers.hoody.com/api/v1/pipe/migration \ | gunzip | psql newdb# 2. Bytes hit psql as they leave FrankfurtCREATE TABLE okCOPY 4823918 okALTER TABLE ok

PUT (curl -T) is preferred because it's how curl wants to upload a stream. POST works identically — same path, same status messages. Use ?n=N on both sides if you need to fan out the same dump to many receivers.

use-cases / move-200gb-between-clouds-with-two-curls / spectator
PROGRESS · ?progress

A spectator URL anyone can watch

A third laptop opens the same pipe URL with ?progress and gets a real-time SSE feed of bytes-per-second, ETA, and connected receivers. Spectating doesn't consume a receiver slot — fifty teammates can watch the migration without changing the n value or interfering with the transfer.

  • NO RECEIVER SLOT
  • UP TO 50 SPECTATORS
  • 30-MIN LINGER
spectator.…hoody.com/migration?progress
SSE · text/event-streamSTREAMING
event: statedata: ["state":"streaming","receivers":1]event: progressdata: ["bytes":83_421_667_840, "speed":"412 MB/s", "eta":"4m 44s"]event: progressdata: ["bytes":118_295_117_824, "speed":"406 MB/s", "eta":"3m 21s"]↻ live · 250ms throttle
use-cases / move-200gb-between-clouds-with-two-curls / why

Why two curls beats six steps

The S3 round-trip looks simple on a whiteboard. In production it's a stack of moving parts that all charge by the second. The pipe collapses the entire stack into the transport itself.

No third storage layer

S3, GCS, Azure Blob — the round-trip exists only because there was no other place to park the bytes. The pipe is the path. There is no bucket to provision, lifecycle-rule, or scrub afterwards.

Pay for flight, not for rest

Egress on the upload, egress on the download — twice. With the pipe the bytes leave Frankfurt and arrive in Singapore in one hop. You're paying for the seconds the connection was open, not for storage you'll delete tomorrow.

HTTP all the way down

Your monitoring already understands HTTP. So does your VPN, your firewall, your audit log. No new IAM identity, no new SDK, no new failure mode — it's a curl command.

DIMENSIONS3 ROUND-TRIPHOODY PIPE
  • Steps6 stages2 curls
  • Disk used200 GB0 bytes
  • Egress legs2 × 200 GB1 × 200 GB
  • Cleanuplifecycle rulenone

Speed is bounded by the slower link end-to-end (Frankfurt egress, Singapore ingress, your TCP window). Hoody's pipe holds zero bytes — there is no server-side storage; backpressure flows directly between the two endpoints.

use-cases / move-200gb-between-clouds-with-two-curls / punchline

Two terminals, one URL, no third storage layer.

The whole migration is the same shape as cat file | wc -l. The fact that the two pipes happen to live in different data centres is an implementation detail of the URL.

  • zero hops
  • zero hold
  • one URL
Read the API
use-cases / move-200gb-between-clouds-with-two-curls / replaces

What this replaces

Anything that exists only because nobody had an HTTP path that streams. The pipe collapses the entire data-migration stack into one curl on each side.

  • AWS S3 cross-region copyTwo egresses, one bucket, lifecycle rules
  • Dropbox transfer200 GB sits on someone else's disk for a week
  • Google Drive hand-offQuota walls, share links, manual cleanup
  • rsync over SSHSSH between random clouds = bastions and key swaps
  • SCP + resumeOne process, no fan-out, breaks on flaky links
  • Third-party data-migration toolsWhole vendor for a curl with extra steps
use-cases / move-200gb-between-clouds-with-two-curls / cta

Skip the bucket. The transport is the URL.

Read the pipe API
use-cases / move-200gb-between-clouds-with-two-curls / related

Read the others