Pular para o conteúdo
#0566
AgentDaemonContainers

Autorestart restarts; the agent repairs

Forty customers, forty worker daemons, autorestart: unexpected on every one. At 2am customer 22's worker starts crash-looping: a config value they changed doesn't parse. The daemon restarts it every few seconds. Restarting doesn't fix a bad config.

A loop on a sentinel agent session runs every ten minutes across the fleet. It reads each daemon's status, and when a program's uptime keeps resetting to seconds it reads that program's stderr log, finds the parse error, corrects the value, then POSTs stop and start. Customer 22's worker comes up clean at 2:11am. The agent files a todo: customer 22 set retry_interval to "5 min"; normalized to 300, consider validating the field.

A supervisor can keep a process alive; only something that reads stderr can keep it healthy.