Ir al contenido
#0999
Files

The quota stopped being every user's problem and became one mount's setting

Twelve analysts, one shared cloud drive, four hundred parquet files, a provider that throttles after a few requests a second.

You mount the drive once, in the analysis container, through the cache backend: POST /backends/cache with the remote, rps: 2, chunk_total_size: "10G", info_age a day, four workers. Every notebook kernel reads from it.

The upstream now sees at most two requests a second no matter how many kernels are open. The first read of a file pulls its chunks in parallel and keeps them on local disk; the second read, from anyone, is local. Listings are cached for the day, so ls doesn't spend quota either.

Nobody wrote a download script or a sync job. The rate limit is a field on the mount, enforced below every process that touches the path.

The quota stopped being every user's problem and became one mount's setting.