Incident response

Building your own AI SRE moves the toil; it does not remove it

Building your own AI SRE moves the toil; it does not remove it

Now that Chronosphere leaders are telling engineering teams to build their own AI SRE, the honest question is who ends up carrying its pager. Their pitch in The New Stack on July 30 is that an in-house AI SRE helps engineers map their systems, investigate incidents, and support reliable software delivery at scale. Chronosphere is now a Palo Alto Networks company, and it sits in the observability lane, so the recommendation lands with the usual caveat about who benefits when telemetry becomes a build target.

The operational read is narrower. An AI SRE that lives inside your own boundary is a second production system. It has a runtime, a data plane, an on-call rotation and a change-management story. If you build it, all of that becomes yours to own alongside the workloads it is meant to keep alive. That is a defensible choice. It is not a shortcut.

What the pitch actually says

The New Stack piece frames the argument as a scale problem. Traditional monitoring tells you a metric moved and hands the rest to a human. An in-house AI SRE, as Chronosphere describes it, absorbs a slice of the follow-up work: correlating signals, walking topology, getting to a plausible root cause faster than a person paging on a Sunday. The claim is that keeping this system in-house lets it learn your services, your naming conventions and your dependency graph in a way a generic external tool cannot.

None of that is unreasonable. Institutional knowledge is what separates a useful runbook from a shelf of PDFs, and the same is true of an incident-response model. Where the pitch understates the cost is on everything after the model runs.

What you own when you own it

Ship an AI SRE inside your own perimeter and the operational balance sheet expands in three concrete ways.

The system needs a permanent access path into production data: metrics, logs, traces and probably deploy metadata. That surface is now something you review, audit and rotate credentials on. It sits next to your pipelines, not away from them.

It also introduces a change-management pipeline for the model and its prompts. Prompt regressions and model swaps behave like any other config change. If your CI already gates Helm charts and Terraform manifests, it now gates whatever glue holds the SRE together, plus its training or fine-tuning steps.

And it needs an on-call of its own. If the AI SRE is in the loop during an incident and it fails, that failure lands on the same rotation that would otherwise be handling the outage. The blast-radius question is real: what does the responder do when the responder is down.

None of this is disqualifying. It is the same shape as adopting any other stateful platform component. It is worth naming because vendor essays tend to describe the benefits and leave the operational balance sheet as an exercise for the reader.

The build-versus-buy line, without the hype

The real argument for building is grounding. A model that knows your service boundaries and your idioms will produce fewer confidently wrong root-cause guesses than one that does not. The real argument against is that most organizations do not have spare capacity to run a second production system whose job is to explain the first one when it is broken.

A defensible middle path exists. Treat the AI SRE as a tool that assists the human on-call rather than one that stands in for them. Keep the failure mode explicit: when the model is unavailable or wrong, the incident process falls back to the same runbook it used before. Log every action the system takes into the same audit trail as your deploys, so a post-incident review can tell the difference between a human decision and a suggested one.

Residual caveat

The Chronosphere read is that this is worth doing at scale. That is probably true for organizations that already run a mature reliability practice and have the headroom to absorb another platform. For everyone else, an AI SRE bolted onto an immature on-call culture inherits the culture, not the pitch deck. The pager still rings, and it still rings for a human.

Source: The New Stack (thenewstack.io)

Related
Incident response

AWS wires its DevOps Agent into PagerDuty incidents

AWS has paired its DevOps Agent with PagerDuty so that the moment an incident is created, the agent runs a root-cause investigation across AWS telemetry, deployment history and a short list of third-party observability tools, posting its findings back onto the incident record.

June 21, 2026
Incident response

AWS DevOps Agent reaches GA with the Datadog MCP Server in tow

AWS has moved its DevOps Agent from preview to general availability, shipping it alongside the Datadog MCP Server so the agent can correlate monitoring signals with AWS-deployed infrastructure during an incident. The badge change is the smaller story; the bigger one is what SRE teams now have to write down before the agent touches production.

June 20, 2026
Incident response

GitHub ships a one-click self-revoke for users whose credentials just leaked

An individual GitHub Enterprise member can now wipe every credential they own, personal access tokens, SSH keys, OAuth tokens and SSO authorizations, in a single action from the Settings page. Containment used to be a scavenger hunt across four menus.

June 25, 2026

Turn this into your pipeline. Build it on Buddy.

Start free