Metrics and analytics · Metrics

Is the North Star moving

Estimate this quarter's untouched resolutions from four inputs, then separate what the product did from what ticket volume did.

estimateeasyExercise · 3 min · 24 of 59

On this page

The quarterly review wants one number and one sentence: how many resolutions without a human touch a week, and whether the team moved it.

The situation

Last quarter the drafting assistant's North Star, resolutions without a human touch per week, was 2,500,000 weekly tickets × 0.60 intent coverage × 0.40 draft acceptance × 0.90 seven-day no-reopen = 540,000 a week, as the North Star lesson works it. This quarter's inputs from the weekly readout: 2,800,000 tickets a week, since two tenants grew and one new tenant went live; coverage 0.66, after two intents were added in week 3; acceptance 0.38, down two points, which the team noticed in week 8 and attributes to the new tenant's longer tickets; no-reopen 0.91. The team shipped the coverage work and a tone fix that was meant to lift acceptance to 0.45. Finance will read the output number; the VP will ask which part the team can take credit for. Assume the product form holds and the four inputs are independent enough to multiply.

The question

Estimate this quarter's weekly output, and say how much of the change against 540,000 is the product and how much is ticket volume.

A strong answer · Rubric · Where answers go wrong

Write your answer first, then open

A strong answer

Multiply: 2,800,000 × 0.66 = 1,848,000; × 0.38 = 702,240; × 0.91 = 639,038. Call it 639,000 a week, up 99,000 on last quarter, 18%.

Separate the two causes by holding tickets at last quarter's 2,500,000: 2,500,000 × 0.66 × 0.38 × 0.91 = 570,570, about 570,600. So the product inputs account for about 30,600 of the rise, 5.7%, and volume accounts for the other 68,400. The sentence for the VP: the North Star is up 18%, of which about a third is the product and two thirds is tenants' growth.

Then the input the team owns most directly. Coverage did its job: 0.60 to 0.66 is a 10% relative lift, worth 2,500,000 × 0.06 × 0.38 × 0.91 ≈ 52,000 at last quarter's volume. Acceptance fell from 0.40 to 0.38, a 5% relative fall, costing 2,500,000 × 0.66 × 0.02 × 0.91 ≈ 30,000, and the tone fix that promised 0.45 did not show. Acceptance gives back more than half the coverage gain, and no-reopen's 0.90 to 0.91 adds about 1%: 1.10 × 0.95 × 1.011 ≈ 1.057, which is why the product's contribution is only 5.7% when coverage alone would have been 10%.

The decision that follows: before claiming the tone fix failed, check whether acceptance fell only for the new tenant. If the other tenants are at 0.42, the fix worked and the blend hides it behind a tenant whose tickets are longer; the cohorts lesson's segmentation is the next query. What would change the estimate: a no-reopen figure measured on a window that has not closed for the newest tenant, which would overstate 0.91.

Rubric

Multiplies the four inputs and reports the output rounded to three figures
Holds ticket volume at last quarter's figure to isolate the product's share of the change
States both shares in the output's unit and as a share of the rise
Prices the coverage gain and the acceptance loss separately and notices the loss gives back more than half the gain
Proposes a segmentation of acceptance by tenant before concluding the tone fix failed

Where answers go wrong

"639,000, up 18%, the quarter was a success" — correct arithmetic, wrong attribution; two thirds of the rise is volume the team did not produce, and the VP's question goes unanswered.

"Acceptance fell, so the quarter was a failure" — one input moved against the team and the output still rose 5.7% on product alone; an answer that reads one input and not the product has not used the form.

"Coverage added 6 points, so the team added 6 points × 2,500,000 = 150,000" — multiplies a rate change by the ticket count instead of through the other inputs; the coverage gain is worth about 52,000, because only 38% of the newly covered drafts are accepted and 91% of those hold.