This first article in our resilience series draws on a conversation with Mark Russinovich about how resilience is changing in the AI era and what it takes to continuously validate it at scale. What surprises me most about resilience failures is how ordinary the drift is. A workload is deployed across availability zones, but a health probe still points to a single dependency.
A database supports failover, but the application’s connection string is pinned to one region. Nothing looks broken. The architecture diagram still shows a resilient design even as the operational reality underneath it changes.
For years, resilience was something you set up once: configure disaster recovery, write a runbook, run the occasional failover test. That kept the lights on, but it treated resilience as a project with an end date rather than a property you maintain. So, when an availability zone or a region has a bad day, the question is not whether a recovery plan exists on paper.
It is whether resilience is still true today, and whether the team can prove it. The dependency that breaks a workload is also changing. On Microsoft’s FY26 Q4 earnings call, Satya Nadella put it plainly: “For whatever reason, if a given model goes away, then you can’t be left high and dry.
You need to be able to still continue your cyber operations.” Traditional disaster recovery planning assumes the critical dependency is infrastructure. Increasingly it is an AI model, an inference endpoint, a retrieval pipeline, or a service operating under capacity constraints.
A workload can be perfectly healthy from an infrastructure perspective and still fail its users because that dependency is unavailable, throttled, or economically impractical to run. That dependency rarely appears on the diagram at all. There is a second shift underneath the first.
An architecture diagram assumes a human drew it and a human will read it. Both of those assumptions are ending, and the dependencies it describes are no longer all deterministic. That changes what it means to know your estate is resilient.
This is the first in a series on how we are helping customers move to resilience that is designed in, measured as the estate changes, and improved over time. We have written before about how to design a resilient workload. This is a different problem: knowing whether hundreds of workloads still match their design today and being able to prove it.
It is where a large share of our roadmap investment is now going. Why resilience drifts Resilience has always been a shared responsibility. We provide the availability zones, a secondary region of choice, and the replication primitives needed to support resiliency, and those do not drift.
What drifts is the other half of the bargain: whether a given workload still uses them the way it was designed to, after a year of changes nobody flagged as risky. Change is where this concentrates. Across the industry, roughly 70 percent of cloud outages are related to change in some way—not dramatic failures, but ordinary modifications whose blast radius nobody re-evaluated.
That is why change discipline matters as much as design. Internally, a change rolls out to a canary region first, then a pilot region, with bake times where health signals are watched before it goes any further. That practice came out of an incident of our own, and it is the same discipline the Well-Architected Framework describes as safe deployment practices.
Learn more about the Azure Well-Architected Framework Disaster recovery is reactive by design. Teams set recovery objectives when a project ships, stand up replication, and then move on, with little ongoing visibility into whether those objectives still hold as the workload changes. Resilience is designed once and rarely revisited.
Across a growing estate of availability zones and regions, the distance between the resilience that was designed and the resilience that actually exists widens quietly, and it usually surfaces only during an incident. We learned this the hard way. Mark tells the story of the storage change that came close to taking Azure down, and what it permanently changed about how we deploy.
What a diagram can’t tell you A diagram is a claim about a system, made once, by someone reasoning about the system as they believed it to be. It is useful, and it is not evidence. There are four things it structurally cannot tell you.
Whether the goal is being met right now. A diagram has no timestamp. Health modeling does: described in the Well-Architected Framework and now available through health models in Azure Monitor , it represents an application as a hierarchy of its components and the signals underneath them, so health is expressed in terms the business recognizes rather than as a wall of resource-level metrics.
Paired with service level indicators, it answers the only question that matters during an incident: is this application meeting its objective right now? The standard that matters is not our own. A service is only healthy if the customer thinks it is healthy—we can believe it is fine, but if the customer is not seeing a healthy service, we have a problem.
What “resilient” even means for this application. On a diagram, resilient is an adjective. A resiliency goal makes it a threshold you either meet or miss and defining it at the level of the application rather than resource by resource is what makes the answer meaningful.
Whether the failover path actually works. Every diagram draws the arrow. Only a test proves it.
The resources nobody drew. A diagram shows what someone remembered. Generated Infrastructure-as-Code covers every resource in the application.
The difference between those two sets is where drift begins—and increasingly the reader on the other end is an agent rather than a person, working from the outcome you asked for instead of the picture you drew. This is how we run Azure.
Originally published at azure.microsoft.com
