A scraping run that survives its own restart

Your crawler is on hour seventeen of a forty-hour run across two hundred thousand product pages. The browser instance dies: out-of-memory after a particularly heavy page, or the process gets OOM-killed, or you push a new container image and something restarts.

The crawl state lives in a SQLite file at /hoody/databases/queue.db: every URL, its status, its scrape timestamp, the parsed payload. The browser writes there after each page. When the instance comes back, it reads the next pending row and resumes.

If the bug is worse than that: half the parsed pages are wrong because of a selector you broke in commit 47a3: you restore the container snapshot from before the bad deploy. The queue is exactly where it was at hour twelve. You re-run the affected five hours.