Specify a campaign-QA agent with a permission boundary
Put each action of a campaign-QA agent on a permission rung, size the human review and audit in minutes, and write the rollback.
designhardExercise · 4 min · 49 of 60
On this page
A vendor's demo shows a QA agent that fixes every ad it flags. The decision is which of those fixes it may make without a person, and how anyone would find out what it missed.
A five-person marketing team launches or edits 200 ads a week across two ad platforms. A coordinator checks each one by hand against a 12-rule checklist: seven rules are mechanical (landing page loads, tracking parameters, headline length, offer end date, price matches the page, countries, daily budget) and five need judgement (unprovable claims, brand tone, image matching copy, endorsement disclosure, competitor mentions). That takes about 10 hours a week, $1,000 at $100 an hour, and last quarter still missed three things: a landing page that returned an error for three days on an ad spending $300 a day, a promotion that ran a week past its end date, and an endorsement ad without its disclosure line.
The head of marketing wants an agent to take over the checklist. The vendor's demo shows it pausing failing ads and rewriting long headlines on its own. Assume 3% of checks fail, a reviewer clears a flag in 1.5 minutes, and one audit check takes 15 seconds.
Specify the agent: put each action it can take on a rung of the permission ladder with the reason, say how you will measure what it misses, and say how any action it takes is undone, keeping human time under 3 hours a week.
The seven mechanical rules run as code, the same answer every time; only the five judgement rules go to a model. 200 ads × 12 checks = 2,400 checks; 3% fail, so 72 flags × 1.5 minutes = 108 minutes of review.
Rungs, by action. Reading ads, pages and budgets is read. Every failed check is a draft: the rule, the evidence and a proposed fix, in the coordinator's queue. Copy, budget and targeting change only with approval, because a wrong rewrite puts unapproved copy live and the platform already truncates a long headline. One action is act: pausing an ad whose landing page fails twice, 10 minutes apart. A dead page converts nobody, so a wrong pause costs hours of delivery, while last quarter's late one cost three days × $300 = $900. It is capped at five a day; a sixth means the site is down, and the agent stops and pages the coordinator.
What it misses lives in the 97% that pass, so each Friday the coordinator audits 10 random passed ads, 5% of the week's 200, on all 12 rules: 120 checks × 15 seconds = 30 minutes. Forty clean ads over four weeks bound the miss rate at about 3 ÷ 40 = 7.5%; each miss becomes a new test case.
Rollback: every action logs the prior state and the rule that fired; one command restores it, and running it twice changes nothing more. A separate write credential means revoking it returns every action to draft.
Human time: 108 + 30 minutes, about 2.3 hours a week. If more than a quarter of flags prove false, fix those checks before promoting any action.
"Give it write access so it fixes what it finds; that is the point of an agent." If one flag in four is false, that is 18 wrong changes a week to live ads, and the coordinator ends up auditing the agent's edits instead of its flags.
"Keep everything at draft, including the pause." Safe, and it would have left the dead landing page spending $300 a day until someone read the queue. The action whose wrong version is cheap and whose delay is expensive is the one to automate.
"Measure the agent by the accuracy of its flags." Flags are the 3% a person already sees. The misses are in the 97% that passed, and only an audit of passed ads finds them; the lesson on agents in marketing operations sizes that audit.
The agent writes to the platform without asking in one branch only.
