Essay argues AI agents should monitor production systems, citing Anthropic's Claude 'superfans'

The post starts from one paragraph in a recent Anthropic blog post, which the author says was not the point of that post. In it, Anthropic's team describes using Claude Tag, its general harness, to monitor each component of its system. They created Slack channels where Claude is tasked with being obsessively focused on the health of a single component, and they call these Claudes "superfans". Examples include a scion-daemon superfan, a scion-cache superfan and an Antique merge queue superfan. The superfans have access to telemetry and logs and act like an experienced engineer with nothing better to do than refresh charts and look for anything strange. Anthropic says it also keeps traditional alerts for all systems, but that the superfans can detect subtle and novel issues that make those alerts seem one-dimensional by comparison.

The author calls this obviously the path forward for operating software. The argument runs as follows. Modern systems are becoming more "grown" than "designed", and as the volume of shipped code rises, the amount of eyes-on-code knowledge any human operator has is falling. Monitoring was never easy: there is a whole industry of observability SaaS companies (Sentry, Datadog, Splunk, Honeycomb are named), observability is expensive, and doing it well takes a specialised skill set that takes years to teach junior engineers. The author says there have been a number of "AI SRE" startups but very little writing from practitioners, and that you do not need a full AI SRE product or a SaaS to improve operations. With Opus 5.5 / Astra class models and a bit of care, the author says it is entirely possible to build a good agent-driven monitoring system that "just works".

The case study is change-point monitoring: intensive monitoring around binary releases and config pushes, since these are the moments most likely to cause a regression. The author quotes the "clichéd" Google SRE book line that "~70% of outages are due to changes in a live system" and says they have worked on the problem in some form since 2023, with little traction. Existing approaches rely on humans to tag metrics diligently and define precise signals, and need significant organisational investment. The author lists why it is hard to get going. Targeting teams with good ops habits mostly sells toil reduction, because their rollouts are already safe, and the system ends up competing with traditional alerts on recall and latency. Opting in all services on universal health metrics gives noisy signals, poor precision and alert fatigue. Before LLMs there was also a precision/latency tradeoff: structured logs are precise but slow, metrics coarse but fast, alerts coarser still.

The author says agents largely solve this, because they will run copious queries, write per-change queries on the fly and scan logs endlessly. In the basic setup the agent just needs access to the codebase and the observability stack. The author has spiked an early prototype: an agent proposes a monitoring plan from the service code, a scaffold executes that plan, and a second agent investigates triggered monitors, with an escalation path to humans. The author says it has detected regressions in systems they have zero understanding of, with precision and latency that would have been impossible a year ago. One design lesson: a pure agent loop is not the best shape. Letting the agent write a monitoring spec that a deterministic system runs gives significantly cleaner signals and dramatically lower token usage. The author places the "secret sauce" in the noise-reduction heuristics, the prompts that get agents to write good specs, and a labelled eval set large enough to validate tweaks.

The post then lists the advantages agents have over human operators. They can track everything around the clock, so any metric regression or suspicious log line can be investigated, not only those above a human "attention cutline". They can fan out across several theories at once; the author already launches a bunch of agents to triage incidents, though that is human-driven today. They are becoming critical for security incidents, where the author points to OpenAI's Daybreak Blue, Anthropic's Project Glasswing and Google's Fairwind as defensive capabilities. And they read code and digest metrics and logs faster. The author concedes that a human with deep knowledge of a system may still find the right signal and root cause faster, but says that to the extent tacit knowledge can be recorded and given to an agent, the agent will eventually win on speed. For systems whose operators are not deeply experienced, time to understanding should be shorter for an agent.

The author then gives uses beyond alerting. Be more diligent: no one would pay a full-time engineer to read every Git commit for a service and set up short-term fast-trigger alerts, but at a couple of cents per million tokens an agent can, and the author expects it to become an industry standard. Continuously improve performance: agents can look at heap dumps, stack traces and CPU profiles and propose optimisations, ideally behind A/B tests. In one instance, a few sessions of Astra/Opus investigation found 50% heap savings and a 30% latency reduction in a common hot-path library that humans had already worked hard to optimise. Take action: agents need not be read-only, and with the right harness and guardrails the author expects them to take front-line remediation actions, from resizing instances to rolling back a config or binary to root-causing bugs and redeploying, while acknowledging that risk grows with the control handed over. On cost, the author argues that Luna/Haiku-class models (meaning GPT-6-Luna and Claude Haiku 5.5) are now competent "graph-starers", and that a moderately large tech company could likely monitor its whole fleet for on the order of $100/day. Finally, the author describes a virtuous cycle in which coding agents let you ship more, monitoring agents let you support more systems, and humans spend more time "building the factory". The conclusion: it will soon be reckless not to have agents monitoring your systems.

In a footnote, the author says a recent set of optimisations by their team saved several million dollars in annualised observability cost, and that this was a mere fraction of what the team spends on observability. Another footnote says general harnesses like Claude Tag evidently already do a good job at agentic monitoring, per the Anthropic post, but that even a little monitoring-specific harness engineering picks up a lot of low-hanging fruit.

Key facts

Anthropic's team uses Claude Tag to run Slack-channel 'superfan' Claudes, each focused on the health of one component (for example a scion-daemon superfan and a scion-cache superfan), with access to telemetry and logs; Anthropic says they catch subtle, novel issues that traditional alerts miss.
The author argues agents suit monitoring because they can watch everything around the clock, fan out across several incident theories, and read code, metrics and logs faster than people.
From an early change-point monitoring prototype, the author reports that a pure agent loop is not the best design: having the agent write a monitoring spec that a deterministic system runs gives cleaner signals and far lower token usage.
The author estimates a moderately large tech company could likely monitor its entire fleet for on the order of $100/day, using Luna/Haiku-class models (GPT-6-Luna and Claude Haiku 5.5).
The author expects agents to move into front-line remediation such as rollbacks, while acknowledging risk grows as more control is handed over, and says it will soon be reckless not to have agents monitoring your systems.

Why it matters

Monitoring is expensive, depends on scarce expertise and takes years to learn. The post argues that agents change this, and it comes from a practitioner who has worked on change-point monitoring since 2023 rather than from a vendor. The author notes there are many 'AI SRE' startups but little writing from people who operate systems. The Anthropic paragraph that triggered it is itself a rare first-hand description of a lab running per-component monitoring Claudes alongside traditional alerts.

Who it affects

Site reliability and platform engineers who run production systems, and the teams that pay for observability. The author names Sentry, Datadog, Splunk and Honeycomb as examples of the observability SaaS industry, and says the approach is cheap enough for startup-scale fleets as well as larger companies. The author also says that as agents take over more of the watching, human operators spend more time building and directing, and that junior engineers currently need years of training in this domain.

How to use it

In the basic setup, connect your codebase (to reason about, say, config consumption) and your observability stack to an agent. The author's prototype has three parts: an agent that proposes a monitoring plan from the service code, a scaffold that executes it, and an agent that investigates triggered monitors and escalates to humans. The author advises having the agent write a spec for a deterministic runner rather than looping on its own, investing in noise-reduction heuristics, prompts for good specs and a labelled eval set. Other uses suggested: reading Git commits to set short-term alerts, proposing performance optimisations from heap dumps, stack traces and CPU profiles (behind A/B tests where needed), and launching several agents to triage an incident. The author stresses that a general harness like Claude Tag already works, and that a little monitoring-specific engineering adds more.

How solid is it

This is an opinion essay built on one paragraph from Anthropic's blog post and the author's own early prototype, not a verified result. No precision or recall figures or benchmarks are given for the prototype. The 50% heap savings and 30% latency reduction is a single observed instance in an unnamed library. The $100/day cost is a hedged estimate ('could likely', 'on the order of') with no method given. The post does not report any incident that agents caught or missed at Anthropic. The Google SRE line about 70% of outages is quoted by the author as a cliché, not as new evidence.

Risks and caveats

The author concedes that a human with deep knowledge of a system may still be faster at finding the right signal and the root cause, and that agents beat people on speed only to the extent tacit knowledge can be recorded and passed to them. A pure agent loop was found to be a poor design, so naive setups may be noisy and token-hungry. Autonomous remediation is described as a future possibility, not something already deployed, and the author acknowledges risk grows with the control handed to the agent. The strong claims, such as 'cost is no longer a limitation' and 'it will soon be reckless not to have agents monitoring your systems', are the author's own judgments.

“It will soon be reckless not to have agents monitoring your systems.”

— From the blog post 'Agentic Monitoring' at benjamincongdon.me