انتقل إلى المحتوى
#0205
BrowserCurl

Scrape with curl until it can't, then escalate

Most pages on your scrape list are static. A GET against hoody-curl returns the HTML, you parse it, you're done. Cheap, fast, no Chromium involved.

Some pages render their content with React on first paint. For those, the curl call gives you a shell of HTML and an empty data div. The pipeline detects the empty div, falls back: same URL, sent to GET /browse?url=... on the same container's browser instance, then GET /text for the rendered page.

Both services live behind URLs on the same container. The fallback is one if-statement and one extra HTTP call. You don't run a Playwright pool 24/7 to handle the 5% of pages that need it. The 95% gets the cheap path.

Use curl until it stops working; the browser is two services away, not two architectures.