Prompt and context engineering on real data · Context assembly

Structured output for the claims form

Design the record that pre-fills the claims platform's triage form: the fields the model writes, the ones it never writes, the checks, and the decline path.

designmediumExercise · 4 min · 37 of 64

On this page

The claims platform pre-fills its triage form from whatever the model returns, and its parser refuses anything it does not expect. The design is the record: what the model writes, what it never writes, and what the handler sees when it cannot decide.

The situation

The claims platform's triage form has four fields the handler confirms: a route code (FT, ADJ or FRD), up to three governing clause references, an exclusion tick-box, and a note of at most 250 characters. The pilot writes one row per claim to pilot.triage_output, and the platform pre-fills the form from it. Its parser rejects a row with an unknown code or an over-long note, showing the handler a blank form; a row with the route left empty is accepted and shows the note. In the prototype the model returned JSON with nothing enforcing it: the route as free text ("fast track", "Fast-track (4.2)") and the clauses as one string read by a regex. Replayed through the platform's parser, 3 of its 50 replies failed, 6%, or 72 blank forms a day at 1,200 claims, and on 1 the regex took a clause number from the reasoning. Each call retrieves three clauses with ids such as 4.2. The output budget is 160 tokens, about 120 words at 0.75 words per token. The definition of done allows 10% of claims, 120 a day, to go undrafted.

The question

Design the record the model returns, field by field, saying what the schema enforces and what code checks, and decide what the handler sees when the model declines or the record fails.

A strong answer · Rubric · Where answers go wrong

Write your answer first, then open

A strong answer

Five fields, in decoding order, evidence before verdict, as the prompt is a spec argues:

{ "clauses": [{ "id": "4.2", "quote": "sudden escape of water from a fixed installation" }],
  "exclusion_applies": false, "route": "fast_track",
  "reason": "Sudden escape from a fixed pipe; no exclusion applies.", "decline": null }

clauses holds one to three objects, an id and a quote of at most 15 words. Code checks each id is one of the three retrieved and each quote appears verbatim in that clause, which catches the citation the regex used to invent. route is fast_track, adjuster, fraud_review or needs_handler; code maps the first three to FT, ADJ and FRD, so a platform code change never touches the prompt. reason is at most 30 words, and code checks the 250-character limit, since decoding often does not enforce length. decline is null or one of conflicting_clauses, wording_not_retrieved, missing_information, set exactly when the route is needs_handler. The schema enforces types, required keys, both enums and exclusion_applies as the tick-box boolean; code checks the rest.

The model never writes the claim id, the term id or the prompt and schema versions; the service stamps them. Three 15-word quotes, a 30-word reason and the keys come to about 100 words, inside the 120 at the prose rate; keys and ids tokenize worse, so measure it on day 1.

A failed check is retried once with the error. A second failure, an API refusal or a decline writes a row with the route empty, the clauses that validated and the reason, so the handler sees what is known rather than a blank form. All three are counted daily against the 120-claim allowance; the prototype's 6%, 72 a day, would use 60% of it.

If the platform can show the clause text itself, drop the quotes and save about 60 tokens.

Rubric

orders the fields so the cited evidence is decoded before the route
confines clause ids to the three retrieved and checks each quote verbatim in code
keeps the platform's codes, the claim id and the versions out of the model's output
gives the model a valid way to decline, with an enumerated reason
separates what the schema enforces from what code checks, including the 250-character note
sizes the record against the 160-token budget and counts every failure against the coverage allowance

Where answers go wrong

"Ask for JSON in the prompt and parse it." That is the prototype: without a schema, an enum and the id and quote checks, the 6% stays, and a record citing a clause that was never retrieved still parses and pre-fills the form with confidence.

"Have the model write FT, ADJ or FRD and the claim id, so the row is complete." The codes belong to the platform, not the wording, and a model copying a claim id out of an adjuster's note attaches the record to someone else's claim.

"Leave out a decline; the handler can always override." Then every undecidable claim gets a route, the agreement figure absorbs the guesses, and nobody can say how much of the 10% allowance the model used, because its declines were never counted.