Guardrails as gates, not filters
Place input, output and action gates on the request path, map each onto an OWASP entry, and state what each costs in latency and where the trace shows it.
On this page
A guardrail written as a line in the prompt — "never reveal another customer's data", "only discuss our products" — is a filter the model applies to itself, and it fails at a rate you can measure. A gate is code at a fixed point in the request path that passes a value or stops it, with a named failure path and a span in the trace. There are three places to put one: before the model, after it, and before any action the model asks for.
Filters and gates
The difference is what the check guarantees. A filter transforms or hopes: a paragraph about safety in the system prompt, a regex that strips swear words from the output, a second model asked "is this reply appropriate?" whose answer nothing acts on. A gate makes a decision with consequences — pass, or stop and take a named path: refuse, retry once, send to a person — and logs its verdict, so its rate is a number on the dashboard.
The prompt injection lesson gave the numbers that make the case. A prompt line holds most of the time — 95% in that lesson's measurement — and a permission check holds every time for the thing it checks. Gates do not make a feature safe in general. They make specific bad outcomes impossible, and say which ones.
The OWASP Top 10 for Large Language Model Applications names those outcomes, and three of its entries map onto the three positions. We cite them by name, since the numbering changed between editions: "Unbounded Consumption" (opens in a new tab), "Improper Output Handling" (opens in a new tab) and "Excessive Agency" (opens in a new tab). Two of the names changed too: the earlier edition's nearest entries were "Model Denial of Service" and "Insecure Output Handling".
Input gates: before the model
An input gate decides whether a request reaches the model at all. Three checks, cheapest first.
Size: a request over the feature's input cap is refused before any tokens are spent. A support bot has no reason to accept a 150k-token paste, which costs 150k × P_in and seconds of prefill on one call. Rate: requests per user per minute and tokens per user per day, enforced as any API enforces a rate limit. Those two are OWASP's unbounded consumption — a user, a script or a looping agent spending your money — and the token cap from agent loops and stop conditions is the same gate inside one task.
Content: a classifier on the user's text that flags out-of-scope requests, abuse, and attempts to instruct the model directly. It can be a small trained classifier at tens of milliseconds, or a call to a small model tier at a few hundred. It has a precision and a recall, measured on its own labelled set like any grader, and a hard limit: it sees the user's input, not the tool results that arrive later, so it cannot stop indirect injection. That is the action gate's job.
Output gates: schema plus invariants
An output gate decides whether the model's output may be used. It is the four checks from reading the response promoted to a gate: stop reason, schema validity, business invariants, provenance. The invariants are code in the constructor of the object the output becomes — constructors and invariants is the same idea without a model — and a failure takes one retry with the failing check named, then a person.
OWASP's improper output handling is the other half: model output passed to something that executes or renders it. Generated SQL run with the application's credentials; a reply rendered as HTML with a script tag in it; a URL fetched because the model wrote it; a shell command assembled from a summary. Model output is untrusted input to the next component, exactly as a form field is. The gate parses and constrains it: SQL parsed and limited to SELECT on allow-listed tables, run as a read-only role with a row limit and a timeout; markdown rendered with links restricted to allowed domains; no model string ever reaching a shell. Milliseconds at most, every time.
Action gates: before the effect
An action gate decides whether a tool call the model proposed may run. Its rules are the tiers from the prompt injection lesson: a per-task allow-list, argument bounds, a schema that cannot express the dangerous case, and a person's approval for anything destructive or irreversible. This is OWASP's excessive agency, and it is the one gate that holds against indirect injection, because it does not care why the model asked. The summariser's allow-list does not contain send_email, whoever wrote the page it read.
The code checks take under a millisecond. Approval takes minutes to hours, so it is never on the synchronous path: the loop pauses, the proposed call joins a queue, and the run resumes with the verdict as the tool result. Approval gates sizes that queue for an automation, and the hooks lesson shows the mechanism that runs the check before every tool call.
Where each sits in the trace
Each gate writes a span with its verdict and latency, so the trace of a refused or retried task says which gate acted and why, and the dashboard carries each gate's rate: input refusals per day, output check failures per thousand calls, approvals requested and granted. A changing gate rate is a signal: output check failures doubling after a deploy is a prompt regression the eval missed, and the trace from the observability lesson names the prompt version.
What the gates cost in latency
| Gate | Check | Added latency | On failure |
|---|---|---|---|
| input | size cap, rate limit | under 1 ms | refuse |
| input | content classifier | 20 to 50 ms trained; 300 to 500 ms as a small-tier call | refuse or route |
| output | stop reason, schema, invariants, provenance | under 5 ms | retry once, then a person |
| output | SQL parse, link allow-list, escaping | under 5 ms | reject |
| action | allow-list, argument bounds | under 1 ms | rejected result to the model |
| action | human approval | minutes | pause and resume |
Two rows cost real time. The output retry costs a whole model call, but only on the few percent of calls that fail a check: at 3%, p95 barely moves while p99 does. The content classifier as a model call costs 400 ms on every request if it runs first. Run it in parallel with the main call instead, and discard the main result when the classifier refuses: no added latency, paid for with the main call's tokens on refused requests. At a 2% refusal rate that is 2% more input spend for 400 ms off every request — usually the right trade on an interactive path, and the wrong one on a batch path where nobody waits.
In an interview
Given a feature, the interviewer wants the gates placed and each one's job named. Say: "an input gate for size, rate and scope — unbounded consumption; an output gate for schema, invariants and anything executable — improper output handling; an action gate with an allow-list, bounds and approval — excessive agency; each a span in the trace with a rate on the dashboard." Then give each gate's latency and say where the classifier runs.
The mistake that loses points is "we added guardrails to the system prompt" as the answer to a safety question. The second is presenting an input classifier as the defence against injection, when the injection arrives in a tool result the classifier never sees.
Check yourself
1. An analytics assistant writes SQL that runs against the reporting database. Which gates go where?
An output gate: parse the SQL, allow only
SELECTon allow-listed tables, add a row limit. An action gate at the database: a read-only role scoped to those tables, a statement timeout, no access to the users table at all. Both are code, and neither asks the model. The prompt can still say "read-only queries"; it is just not the control.
2. The input classifier is a small-tier call with a 400 ms p95. The latency budget is tight and 3% of requests are refused. Sequential or parallel?
Parallel. Sequential adds 400 ms to every request; parallel adds nothing and wastes the main call's tokens on the refused 3%, so input spend rises about 3%. Run it first only if the main call has side effects that cannot be discarded, because then the gate must come before them.
3. One user's script sent 2,000 requests of 100k tokens overnight. Which gate was missing, and what numbers would you set?
The unbounded-consumption gate: 200M input tokens, 200M ×
P_in, on one account. A per-request cap near the feature's real need — a support message rarely exceeds 5k tokens — would have refused every one. Add a per-user daily token budget at a few times the heaviest legitimate user's, and alert on cost per user as well as per task.