Prompt evaluation and optimization

"It looked better" isn't evidence

Most prompt tweaking goes like this: try a prompt, read one output, change a few words, read another output, decide it "feels better". The problem is that model outputs vary from run to run, and one example tells you very little. You might be improving one case while breaking three others you didn't look at.

Prompt evaluation replaces vibes with a repeatable check: the same inputs, the same criteria, compared across versions.

Step 1: Define what "good" means

Before testing, write down the success criteria for your task. Be specific and, where possible, checkable.

Some criteria are objectively checkable (bullet count, valid JSON, contains a required field). Others need judgement (tone, helpfulness). Know which is which, because you'll check them differently.

Step 2: Build a test set

A test set is a collection of realistic inputs you'll run every prompt version against. It doesn't need to be huge to be useful — even 15–30 well-chosen cases will catch many problems. Include:

• Typical cases — what most real inputs look like.
• Edge cases — very short, very long, missing information, multiple languages, unusual formatting.
• Tricky cases — inputs that previously caused failures, or that contain instructions ("ignore the above…").
• Expected outputs or notes, where possible: the correct category, the facts that must appear, things that must not appear.

Real (appropriately anonymised) inputs beat invented ones. Every time you find a failure in production, add it to the test set.

Step 3: Score with a rubric

A rubric turns your criteria into scores. For each test case and each criterion, record pass/fail or a small scale (1–3 works well; wide scales tend to be scored inconsistently).

case_id | version | length_ok | main_issue | refund_flag | no_invented_facts | tone
--------+---------+-----------+------------+-------------+-------------------+-----
  07    |   v1    |   pass    |   pass     |   FAIL      |   pass            |  2
  07    |   v2    |   pass    |   pass     |   pass      |   pass            |  3

How to score:

• Code checks for anything objective: length, format, JSON validity, required keywords, exact-match labels.
• Human review for judgement-heavy criteria, ideally with the rubric in front of you and without knowing which version produced the output.
• Model-graded checks ("LLM as judge") can scale up review: a separate prompt grades outputs against the rubric. They're useful but imperfect — graders can be biased toward longer answers or their own style. Spot-check them against human judgement before trusting them.

You are grading a summary against a rubric. Grade only what is asked.

<email>{{original_email}}</email>
<summary>{{model_summary}}</summary>

For each criterion, answer "pass" or "fail" with a one-line reason:
1. Contains no facts absent from the email.
2. States whether a refund was requested.
3. Uses neutral tone.

Return JSON: {"no_invented_facts": ..., "refund_flag": ..., "tone": ..., "reasons": [...]}

Step 4: Compare versions fairly

• Change one thing at a time. If you rewrite the instructions, add examples and change the output format at once, you won't know which change helped — or which one quietly hurt.
• Run the whole test set for each version, not just the case you were trying to fix.
• Account for variability. For important decisions, run each case more than once and look at the pass rate, not a single output.
• Keep model and settings fixed while comparing prompts. Changing the model, temperature or other parameters is a separate experiment.
• Version your prompts. Keep each version with its results, so you can roll back.

Optimising: what to try

Once you can measure, common levers to test (one at a time) include: clearer task definition, explicit output format, adding or changing examples, moving long context before the question, adding a rule for a specific failure, splitting the task into steps, or asking for checkable intermediate work. Let your failure cases suggest what to try: if most failures are invented facts, work on grounding; if they're format errors, work on structure.