コンテンツにスキップ
#0998
Files

The bucket has a size limit. The path doesn't

The training set is one 40 GB file, and the object store behind it caps a single object at 5 GB.

POST /backends/chunker on the files service, pointing at that remote: chunk_size 5 GB, hash_type: sha1, fail_hard: true. Mount it. At the path there is one file, 40 GB, readable with open() like any other. On the far side there are eight objects named by the chunk pattern and a metadata record carrying each chunk's checksum.

fail_hard is the flag that matters: a file with a missing or invalid chunk becomes an error, not a partial read. A truncated dataset that loads without complaint is worse than one that refuses to.

The application never learned the ceiling existed. The mount split the file on the way out and reassembled it on the way in.

The bucket has a size limit. The path doesn't.