跳转到内容
#0475
AgentDaemonContainers

Route the cheap turns to a model on the box

Most of what your agent does is not hard. Classify the email. Extract the date. Decide whether this turn needs the expensive model at all. Eighty percent of calls, small model, trivial prompts.

Those calls still cost tokens if they leave the box. So they don't.

An Ollama server runs as a daemon program in a container on your Hoody server, serving a small open model from the box's own cores. The agent's cheap provider profile points at it as an OpenAI-compatible endpoint on localhost. The hard turns go to a second profile with your real Anthropic key, billed by Anthropic, no Hoody markup. Swapping profiles is a configuration change.

The easy 80% now costs what the server costs, a flat rate you were paying anyway. The hard 20% is the only token bill.

The server you rent is also an inference provider, and it doesn't send an invoice.