Weekly reporting
Design a client report that goes out at 08:00 Monday with nobody reading it first, and show the arithmetic that lets code checks stand in for the reviewer.
On this page
Every Monday at 08:00 each of the agency's 12 retainer clients gets one report: a table of figures from six sources and three paragraphs on what moved, why, and what happens next week. By hand an account manager spends 45 minutes on each, 9 hours a week. The constraint that shapes the design is "no human input": the account managers start at 09:00, so nobody reads a report before the client does. That makes the code checks the gate, and most of this page is about what earns them the job.
The role
The six fields of the virtual employee frame, for a task module 1's first exercise scored 300, third of six, with the most hours and six connectors to build.
| Field | Weekly reporting |
|---|---|
| Inputs | Six sources per client, read through Sunday 23:59: web analytics, search performance, two ad platforms, the email platform, and the project tool's delivered tasks. The client's goals and last week's table. |
| Outputs | One document per client: 12 figures with week-on-week change, three paragraphs of at most 250 words, and the week's delivered work. Code computes every figure; the model writes prose only. Never a figure from a source that failed its checks. |
| Done | By 08:00 Monday, 12 reports sent, each complete or with named sections marked "to follow by 12:00", and every withheld section with its account manager. |
| Manager | The operations manager: owns the six connections and the checks, runs the weekly review. Account managers own each client's goals and the withheld sections. |
| Escalation | A source failing its checks: that section is withheld and its account manager has it at 09:00. The run not finished by 06:30: the owner, paged. A credential within 14 days of expiry: the owner. |
| Budget | About $0.01 a report in tokens. People: about 3.75 hours a month, none of it before 08:00. |
The SOP
Trigger Monday 05:00, the agency's timezone; a dry run Sunday 20:00.
Steps 1. Pull the week from each of six sources, per client. code
2. Check each pull. code D1
3. Compute the figure table and week-on-week changes. code
4. Write three paragraphs from the table. model
5. Check the prose against the table; retry once. code D2
6. Assemble; mark withheld sections "to follow by 12:00". code D3
7. Send at 08:00, keyed on client and week. code D4
Decisions D1 a pull passes = data through Sunday 23:59; all 7 days present; each
figure within ±50% of its 4-week average, or a
project-tool note explains the move
D2 prose passes = every number in it appears in the table; every
"rose", "fell" or "flat" matches its change's sign;
no figure from a withheld section; 250 words at most
D3 withhold = a pull that fails D1 drops its rows and any paragraph
citing them; the rest of the report still goes
D4 send = one report per client per week, however many reruns
Exceptions A pull failing D1. Prose failing twice. The dry run not matching last
week. A credential expiring within 14 days.
Done By 08:00 Monday, 12 reports sent, each complete or with named sections
to follow, and every withheld section with its account manager.
D3 is the rule the whole design leans on: degrade, never guess. A report with one section marked "to follow" keeps the 08:00 promise and says nothing false.
The mistake: the model did the arithmetic
The first version handed the model each client's six raw exports and asked for the whole report, figures and prose. The operations manager read all 48 reports from its first four weeks afterwards and found two wrong. One said conversions "rose 12%" when they went from 25 to 21, a 16% fall: the model had compared the wrong columns. The other reported zero email opens and wrote "engagement fell sharply". The email platform's token had expired, the connector returned an empty export with no error, and the model explained the zero instead of noticing it, the silent failure drift and silent failure warns about.
Two in 48 is about 4%. Module 1's exercise priced a wrong client report at $200 of credibility, and approval gates gives the test:
first version 4% × $200 = $8.00 a report
review 5 minutes × $30 ÷ 60 = $2.50 a report
break-even $2.50 ÷ $200 = 1.25%
At 4% every report needs a reader, and a reader means 10:00, not 08:00. Forty-eight reports cannot pin a rate, but they showed the mechanism. Both errors came from asking the model to do work code does exactly: subtraction, and noticing an empty file.
The fix moves every number out of the model. Code computes the table; the model writes prose from it; code checks every number and every direction word in the prose against the table. The expired token now fails D1's completeness check and withholds one section, instead of becoming a sentence.
What a reader would still catch, a sentence accurate in every figure but misleading, is assumed at 1 report in 200: 0.5% × $200 = $1 a report, under the $2.50 a read costs. "No human input" is earned by that arithmetic, and the assumption is the evidence to watch.
The rung, and why not the one below or above
This is a rung 2 build: a fixed sequence on a schedule, 72 pulls a run, two checks written as code, and an owner who edits a canvas.
The trigger is where the first silent bug hides. n8n's Schedule Trigger fires in the workflow's timezone if one is set and otherwise in the instance's (Schedule Trigger (opens in a new tab)). A self-hosted server left on UTC fires hours off local time, and moves again at a daylight-saving change. Set the workflow's timezone, and alarm if the run has not finished by 06:30.
Why not rung 1. A trigger-action chain would loop 12 clients through six reads each, and its two checks are the only gate a client's report passes. On rung 1 neither check is a reviewable diff, and nothing runs them before Monday.
Why not rung 3. The recruiting screen climbed to rung 3 because applications arrive at any hour, leaving no gap between an edit and live traffic. This automation runs once a week, so it has that gap. Every Sunday at 20:00 a dry run replays last week's frozen pulls through the whole workflow; the figure tables must match last week's exactly and the prose must pass D2. A mismatch holds Monday's run and pages the owner. That is a test before deploy, on the canvas, with no developer: the $160 a month retainer would be 14% of the task's whole Sum 1.
Why not rung 4. An agent would spend turns choosing the order of six reads that never changes, and it would do the arithmetic, the first version's failure. Its tokens, by the per-turn arithmetic, with a 2,500-token start (1,500 of instructions and goals, 900 of tool schemas, 100 of task) and about 700 more each turn:
9 turns 9 × 2,500 + 700 × (0 + 1 + … + 8) + 100 × 9 = 22,500 + 25,200 + 900
= 48,600 in and out, plus 400 of report
workflow 2,800 in + 400 out = 3,200, about a fifteenth
About $0.11 a report against $0.01, $5.50 a month against $0.50. The money is not the objection; the arithmetic is.
The graph
Every dashed path leads to an account manager after 09:00, never before 08:00, and none stops the report: a withheld section travels as "to follow", and the rest goes on time.
The gates
There is no human gate, by the arithmetic above, so the two checks carry the weight and are built to fail loud.
The pull check, D1, catches the empty export, the week with five days in it, and the figure that halved for no recorded reason. Its ±50% band is wide on purpose. A real 40% fall should reach the client with its paragraph; the band exists for the data error that makes a source report zero. A project-tool note such as "campaign paused Wednesday" lets a large move through, because the cause is on record.
The prose check, D2, is deterministic, the same shape as the figure check in deterministic first: extract every number from the prose, find it in the table; find every direction word, compare it with the sign of the change it describes. It cannot judge whether a sentence is wise. It can say every time whether "rose 12%" is true.
The reader who is not a gate. The operations manager reads one sent report a week, rotating through the clients, about 10 minutes. Thirteen reads a quarter cannot measure a rate of 1 in 200, since they expect 0.07 misleading reports. That makes a single misleading report found in a quarter strong evidence the assumption is wrong. If it happens, the reports go at 10:00 after an account manager reads them, until the eval set shows why.
The connectors
Six sources, each read through one agency-level, read-only grant that covers all 12 clients' accounts, so six credentials rather than 72. Each identity is scoped to reporting reads and nothing else, rotated every 90 days, with an alarm 14 days before any expiry, the practice in least-privilege credentials.
A run is 72 pulls of about 3 pages each, roughly 216 requests. Assume the strictest source allows 30 requests a minute; its 36 requests take about 72 seconds. The whole run, including 12 model calls, takes about 10 minutes, so a 05:00 start leaves 80 minutes for retries with backoff before the 06:30 alarm and nearly three hours before 08:00, the budgeting rate limits, retries and backoff teaches.
Two details stop duplicates and stale numbers. Ad platforms keep revising recent days as late conversions arrive, so the table says "as of 05:00 Monday", and next week's table restates any figure that moved by more than 2%. The send carries the key client plus week, so a rerun after a crash cannot send a second report, and a withheld section sent at 11:00 goes as a follow-up to the same thread, the idempotency of module 5.
The budget
The two sums from how to use this course, at the assumed $30 an hour for staff and $100 for the founder, with 12 × 4.33 = 52 reports a month.
Sum 1. 12 × 45 minutes = 9 hours a week, × 4.33 = 39 hours × $30 = $1,170 a month.
Sum 2, once. 36 founder hours: six connections at about 4 hours each with their per-client mapping, 6 for the table and the two checks, 4 for the prompt and template, 2 for the dry run. $3,600.
Sum 2, run. At the assumed $2 per million input tokens and $8 per million output, a report is 2,800 in and 400 out: $0.0056 + $0.0032 = $0.0088, about $0.01 with retries. 52 reports are $0.50 a month, $1 with the Sunday dry run. With the assumed $20 platform, $21.
Sum 2, people.
withheld sections 4.3 a month × 15 min 65 min
prose fails twice 1 a month × 10 min 10 min
sampled read 4.33 × 10 min 43 min
credential rotation 2 a month × 10 min 20 min
weekly review 4.33 × 20 min 87 min
total 225 min = 3.75 h × $30 = $113
Sum 2 is $21 + $113 = $134 a month. The saving is $1,170 − $134 = $1,036, and the $3,600 build pays back in 3.5 months.
Reading every report first would add 52 × 5 minutes = 4.3 hours, $130 a month, to remove about $1 × 52 = $52 of expected loss, and it would move the reports to 10:00. The build cost is the input this decision turns on: six connectors are 24 of its 36 hours, so a connector that takes 12 hours instead of 4 moves payback to about 4.3 months, still inside the year.
The evals
The labels already exist. For the eight weeks before launch the account managers wrote these reports by hand, so the last 8 weeks' frozen pulls, × 12 clients = 96 cases, come with the figures that were actually sent. Where the code's table and the sent report differ, the account manager decides which was right, and that answer becomes the label. What "working" means asks for traffic from the automation's own work, and these are those same 96 reports.
Three kinds of case, graded by code as evals as the unit test of LLM work describes:
96 real weeks figure tables match the labels 96 of 96
first drafts pass D2 86 or more of 96
drafts still failing after the retry 0
20 planted drafts a flipped sign, a swapped figure, a 20 of 20 caught
number from a withheld section
6 broken pulls one empty or stale pull per source 6 of 6 withheld
a run 96 × $0.0097 about $0.93
The planted drafts are the point. They measure the gate, not the model: a prose check that misses a flipped sign is a gate that is not there, whatever the model's own error rate. They need no model call, so they cost nothing to run, and all 20 must pass on every change to the prompt, a check, a connector or the model, and in every Sunday dry run.
Between runs, the rates from drift and silent failure watch the live system: withheld sections a week, normally about one, alarming at three; the share of first drafts failing D2; the null share per source per client; and the time the run finishes. The weekly review reads them in its two minutes on the eval trend.
In practice
Tell the clients what the report promises: "Every figure is computed from the source and checked before it is sent. If a source is late, its section says so and follows by noon." Tell the account managers that a withheld section is the system working, and that the one thing they never do is type a figure into a sent report by hand.
The mistake that costs money is letting the model compute the figures. A wrong "rose 12%" reaches a client at 08:00 with nobody between, and the only fix for an untrusted report is a reader, which costs the 08:00 promise.
Check yourself
1. A new client's contract says a named person must check every figure before the client sees it. Does the design change for everyone?
No, only for that client. Its report pauses at a gate after step 6: an account manager on a rota reads it at 07:30, about 5 minutes, $2.50 a week, and signs before 08:00. If nobody signs, the report is held and a one-line note says it follows by 10:00; it is never sent unsigned. The other 11 keep the arithmetic: $1 of expected loss against $2.50 of review.
2. The Sunday dry run matches last week's figures, but 3 of the 20 planted drafts now pass D2 after a teammate "simplified" the number extraction. Run Monday?
Not with that check. A prose check that misses 3 of 20 planted errors is not the gate the arithmetic assumed, and nobody reads before 08:00. Revert the check and rerun the 20. If it cannot be fixed by 05:00, an account manager reads every report and they go at 10:00, about 60 minutes of reading, $30, which is cheaper than one wrong figure at $200.
3. The agency signs 12 more clients, 24 in all. Estimate what changes in the run, the money and the people.
The run is 144 pulls, about 430 requests; the strictest source's 72 take about 2.4 minutes, so 05:00 still leaves hours of slack. Tokens double to about $1 a month, $2 with the dry run. Withheld sections and second failures double, to 130 and 20 minutes; the review, the sampled read and the six credentials do not change. People come to about 300 minutes, $150, and run to $22, so against a Sum 1 of $2,340 the saving rises to about $2,170 a month.