NFRs Are Not Documentation. They Are the Reason Your AI Can Be Trusted.

The most consequential AI failures will not come from the model. They will come from the foundation underneath it.

On a Friday in July 2024, a single configuration update brought down 8.5 million computers worldwide.

Airlines grounded flights. Hospitals postponed surgeries. Banks went dark. Emergency services lost dispatch systems. The estimated financial impact ran into billions of dollars within hours.

The cause was not a sophisticated cyberattack. It was not a complex architectural failure. It was a faulty content update to a security platform that millions of systems depended on. A single non-functional weakness in a foundational component, the way an update was validated and deployed, cascaded into one of the largest IT outages in history.

What struck me most about that incident was not the scale. It was the mechanism. Millions of organizations had built their operations on top of a platform, trusting its non-functional properties. The reliability. The deployment safety. The failure isolation. When those properties failed, every system above them failed too, regardless of how well each individual system had been built. The applications were fine. The foundation was not.

I have been thinking about that incident a lot as organizations rush to deploy autonomous AI. Because AI systems have exactly the same dependency on the non-functional foundation beneath them. And most organizations are paying far more attention to the intelligence at the top than to the foundation that determines whether that intelligence can be trusted at all.

This week I want to talk about non-functional requirements. Not as a pre-launch checklist. As the structural foundation that determines whether your AI can be trusted.

Why NFRs are the hidden foundation of trustworthy AI When organizations talk about trustworthy AI, they almost always talk about the model. Is it accurate? Is it fair? Is it explainable? Is it aligned?

These are real questions. But they assume something that is rarely true: that the platform underneath the model is solid.

An AI system does not operate in isolation. It operates on top of a platform. It consumes data through APIs that have latency characteristics. It makes decisions that depend on services with reliability characteristics. It produces outputs that flow through pipelines with observability characteristics. Every one of these non-functional properties shapes whether the AI system as a whole can be trusted. The CrowdStrike incident is the macro version of this. A foundational platform's non-functional failure cascaded into every system above it. The micro version happens inside enterprises every day, invisibly, in AI systems.

Consider an autonomous agent that makes time-sensitive decisions. Fraud detection. Trade execution. Real-time pricing. The agent's logic might be flawless. But if the API it depends on has unpredictable latency, the agent will sometimes make decisions on stale data and sometimes time out entirely. The model did nothing wrong. The non-functional foundation failed it.

Consider an agent that requires a complete view of context to make a correct decision. If the underlying services have inconsistent reliability, the agent will sometimes operate with partial context, unaware that some of its inputs failed silently. The decision it makes will be confidently wrong, and nothing in the model will flag the problem.

NFRs are not a checklist you complete before launch. They are the structural properties that determine whether the intelligence you build on top can be trusted to behave consistently. Get them wrong, and you are building sophisticated reasoning on an unstable foundation. The CrowdStrike outage showed the world what happens when that foundation fails at scale.

The four NFRs that matter most for AI systems Not all non-functional requirements are equally consequential for AI. Four matter more than the rest because AI systems depend on them in ways human-operated systems do not.

The first is latency predictability. Not just low latency. Predictable latency. An AI agent that operates across multiple reasoning steps, each one calling a service, accumulates latency at every step. If each underlying service has unpredictable response times, the agent's total execution time becomes wildly variable. For time-sensitive decisions, this variability is the difference between a useful system and a dangerous one. The platform foundation must guarantee not just speed but consistency of speed. The second is reliability with explicit failure semantics. When a service an agent depends on fails, what happens? Does the agent receive a clear error it can reason about, or does it receive a partial response it mistakes for complete? Human engineers handle ambiguous failures through judgment. Agents cannot. The platform must provide explicit, unambiguous failure semantics so that an agent always knows whether it has complete information or not. Silent partial failures are the enemy of trustworthy autonomous systems. The CrowdStrike incident was, at its core, a failure of safe deployment and failure isolation. The same class of weakness, applied to the services your agents depend on, produces silent agent failures rather than visible global outages.

The third is observability at the decision level. Standard observability tracks system health. Logs, metrics, traces. For AI systems, observability must go further. It must capture not just what the system did but what the agent saw when it decided. The inputs available at decision time. The data freshness at that moment. The services that responded and the services that did not. Without this decision-level observability, you cannot reconstruct why an agent made a particular decision, which means you cannot govern it. The fourth is security with data provenance. An AI agent consuming data needs to know not just that the data is correct but where it came from and whether it can be trusted. Security for AI platforms is not just access control. It is provenance. Can you trace every piece of data an agent consumed back to its source through a governed, auditable chain? Without provenance, an agent can be manipulated through its inputs in ways that are invisible to traditional security controls. Why deployment safety matters more for AI than for anything before it The CrowdStrike outage was fundamentally a deployment safety failure. An update reached production systems without sufficient staged validation. The blast radius was unbounded because there was no mechanism to limit how far a bad update could propagate before it was caught.

This is precisely the risk profile of autonomous AI systems, amplified.

When you update a model, change a prompt, modify an agent's tool access, or alter the data pipeline an agent depends on, you are deploying a change to a system that will then make autonomous decisions at scale. If that change has an unintended consequence, the blast radius is every decision the agent makes until someone notices.

The non-functional discipline that prevents global outages is the same discipline that prevents autonomous AI incidents. Staged rollout. Canary deployment. Automated rollback on anomaly detection. Blast radius limitation. These are not just operational niceties. For autonomous AI, they are the difference between a contained incident and an unbounded one. At organizations where I have seen AI deployed safely, the platform team treats every change to an AI system with the deployment rigor they would apply to critical infrastructure. Changes roll out gradually. Behavior is monitored against a baseline at each stage. Anomalies trigger automatic rollback before the change reaches the full production fleet. At organizations where I have seen AI deployed dangerously, changes to models, prompts, and agent configurations are pushed with far less discipline than they would apply to a database schema change. The assumption is that because the change is to the intelligence layer, it is somehow lower risk. The CrowdStrike incident is a reminder that the foundation is always where the largest blast radius lives.

Your 30-day and 90-day plan 30 days: Audit your NFR foundation In the next 30 days, the goal is to understand the non-functional foundation your AI systems actually run on.

Pick your most consequential AI system. Identify every service it depends on. For each service, document the four NFRs honestly.

What is the latency distribution, not the average but the variability? What are the failure semantics, and does the agent receive explicit errors or potentially silent partial responses? What is the observability depth, can you reconstruct what the agent saw at decision time? What is the data provenance, can you trace every input to its source?

Then audit your deployment safety. When you change this AI system, how does the change reach production? Is there staged rollout? Canary deployment? Automated rollback? Or does a change go to the full fleet at once, the way the CrowdStrike update did?

For most organizations, this audit reveals that the non-functional foundation is inconsistent and the deployment discipline for AI changes is weaker than for traditional infrastructure. Document where those gaps are. They are your highest-priority remediation targets.

90 days: Make NFRs and deployment safety enforceable defaults With the audit complete, the 90-day goal is to move NFRs and deployment safety from aspirational guidelines to enforceable defaults.

For the four critical NFRs, establish them as standards that every service supporting an AI system must meet. Predictable latency. Explicit failure semantics. Decision-level observability. Data provenance. Where services fall short, prioritize remediation by the consequence of the AI decisions that depend on them. For deployment safety, implement staged rollout and automated rollback for every change to an AI system. Model updates. Prompt changes. Agent configuration changes. Data pipeline changes. All of them should reach production gradually, with behavior monitored against a baseline and automatic rollback on anomaly. Establish blast radius limits. No change to an autonomous AI system should be able to reach the full production fleet without passing through a staged validation that would catch an unintended consequence before it propagates. At 90 days, your most consequential AI systems should run on a foundation where the four critical NFRs are enforced by default and where no change can produce an unbounded blast radius.

The question for your next platform review If a change to your most critical AI system had an unintended consequence today, how far would it propagate before you caught it?

If the answer is the entire production fleet, you have a CrowdStrike-shaped risk in your AI estate.

That is the gap to close.

Next issue

We have covered data ownership, domain boundaries, integration patterns, and now the non-functional foundation. Four structural prerequisites for trustworthy AI.

Next issue I want to address what happens when these foundations are solid but the organization still cannot see what its AI is doing. The observability gap, and why most organizations monitor their AI systems at the wrong level of abstraction entirely.

Platform Path · Issue 06 The Observability Gap: Why You Monitor Your AI at the Wrong Altitude

Until next week.

Rupali