Every AI agent demo works. That’s what makes demos a bad predictor of whether the thing survives contact with real users.
The gap between “works in the demo” and “safe to run in production” isn’t more prompt engineering. It’s three layers most teams build in the wrong order, or skip entirely: guardrails that stop bad behavior before it happens, evals that prove a change didn’t quietly make things worse, and observability that tells you what actually happened when — not if — something goes wrong anyway.
Guardrails: rules the model can’t argue with
The instinct is to put safety instructions in the system prompt. It feels like it should work, and in the demo, it does. The problem shows up the first time a user — accidentally or on purpose — sends a message that talks the model out of its own instructions. This is prompt injection, and it works because a system prompt and a user message are both just text to the model; there’s no privileged layer separating “instructions” from “input.”
Guardrails fix this by moving the check outside the model entirely. An input guardrail runs on the user’s message before any LLM call happens — rejecting off-topic requests, redacting PII, catching injection attempts. An output guardrail runs on the model’s response before a user ever sees it — validating structure, length, and whether the claims are actually grounded in what you gave it. Code either runs a check or it doesn’t; there’s no version of a regex that can be persuaded to skip itself.
Evals: the thing that has to be able to fail
A team that ships a prompt change without running it against a fixed test set is trusting vibes. Evals are the fix — a golden dataset of representative input/output pairs, scored automatically, run before anything ships. The number that matters isn’t the size of the dataset (though below roughly 50 examples you can’t distinguish a real regression from ordinary variance) — it’s whether the eval is capable of failing.
This sounds obvious and gets skipped constantly. A team builds an eval, watches it pass, ships the change, and never checks what happens when the model genuinely gets something wrong. An eval that’s never seen a failing case isn’t validated — it’s decoration with a green checkmark. The useful version of an eval is the one you’ve deliberately broken at least once, so you know it actually notices.
Observability: what happened, and whether it’s getting worse
Guardrails and evals both assume you know what to check for in advance. Observability is for everything else — the failure mode nobody anticipated, and the slow decline nobody would notice call-by-call.
There are three pieces to this, and they answer three different questions. Tracing answers “what happened in this one specific call” — a structured log of every step, every tool call, every latency number, replayable after the fact. Metrics answer “is this getting worse system-wide” — cost, latency, and failure-rate trends dashboarded and alerted on. Human feedback answers the question the other two can’t: whether quality is quietly drifting downward in a way that never shows up as an outright error. A 1% weekly decline in response quality generates zero error logs and zero pages — and compounds into a genuinely bad system in a few months if nothing is watching for it.
Why the order matters
Teams that build observability first and guardrails last end up with excellent visibility into a system that was unsafe the whole time. Teams that build guardrails and stop there ship something safe against the failure modes they thought of, with zero signal on the ones they didn’t, and no mechanism to catch the version of the prompt that quietly got worse. The three layers aren’t optional extras stacked on top of a working agent — they’re what makes “working” a claim you can actually defend six months later, not just at launch.
If you want to build a small version of this stack yourself — two guardrail functions and an eval you can watch genuinely fail — Chapter 7 of the AI Transition Course walks through it hands-on, alongside the 6-beat interview answer for explaining this to a hiring panel — see the companion interview-questions piece for that side of it.