Evaluating & Iterating Prompts

Introduction

Module 1 introduced the loop — ASK → READ → JUDGE → REFINE — and left the judging and refining deliberately shallow. Module 4 then raised the stakes: once a prompt becomes a pattern, its output starts to look familiar, and familiar output is much easier to stop reading carefully.

This module makes JUDGE and REFINE into real skills.

How do I tell whether this result is actually good enough? And when it is not, what exactly do I change?

Good Sounding Is Not Good Enough

The most common evaluation failure is being persuaded by tone. A confident, well-organized, fluently written answer can address a slightly different question than the one you asked — and read as though it succeeded.

Fluency is a property of the writing. Fitness is a property of the match between the result and the task you set. Only one of them is what you needed.

The Loop, Taken Seriously

ASK → INSPECT → IDENTIFY THE PROBLEM → CHANGE ONE THING → TRY AGAIN → YOUR JUDGMENT.

Module 1 taught this as four steps because a beginner needed the shape. Here it grows two, and both additions are the point of the module. INSPECT is separated from ASK because reading properly is a distinct act that people skip. IDENTIFY THE PROBLEM is separated from CHANGE because naming the weakness before editing is what turns a retry into an experiment.

The last step never moves. The loop produces candidates; you decide.

Seven Questions Worth Asking

Plain language, in the order a person naturally checks them.

1. Did it do the task? Not "is it good" — did it do the thing you asked for?
2. Is it accurate enough to use? Which claims would matter if they were wrong?
3. Is anything important missing? Absence is harder to notice than error.
4. Is the format usable? Could you act on it as it stands, or must you rebuild it?
5. Does it fit the audience? Right content, wrong reader, is still a failure.
6. Did it respect the constraints? Length, tone, what to avoid.
7. Can I verify what matters? If you cannot check the load-bearing claim, that is the finding.

Most weak results fail two or three of these, and naming which ones is already most of the diagnosis.

Where Did It Go Wrong?

Module 2 gave a component locator for role, instruction and example. This extends it across everything the course has taught, and adds the row people most often need.

• •Did the wrong work → the instruction: the task was not stated precisely enough.
• •Right work, wrong assumptions → context: it was never told something it needed.
• •Right content, wrong voice or depth → the role or the audience.
• •Right substance, unusable shape → the output format or the example.
• •Ignored a rule you take for granted → a constraint you never wrote down.
• •Confidently states something untrue → verification. This is not a prompt problem.

A fluent false statement is not a phrasing failure, and rewording will not fix it. It is a signal to check the claim yourself, or to supply the source.

Change One Thing

When you change three things and the result improves, you have learned nothing about which change helped — and you now carry two edits you cannot justify.

Change one meaningful thing, and say in advance what you expect it to fix. If the result changes in a way you did not predict, that is more informative than a vague improvement: it usually means your diagnosis was wrong, which is worth knowing before you build a pattern on it.

Two honest caveats. The same prompt can produce somewhat different answers on different attempts, so a single comparison is evidence, not proof. And "one thing" means one meaningful thing — fixing a typo and adding a constraint at once is fine; adding a constraint and changing the audience is not.

When the Prompt Is Not the Problem

The most valuable judgment in this module is knowing when to stop editing the prompt.

• •It does not know. The information was never available to it, and no phrasing creates it.
• •The source is insufficient. You supplied material that does not contain the answer.
• •The task is not appropriate. It needs a decision, an authority or an accountability that belongs to a person.
• •It needs real expertise. Medical, legal, financial, safety-critical or regulated questions need a qualified human, not a better brief.

Rewording a prompt five times is itself a diagnosis. If clarity is not the bottleneck, more clarity will not help — and continuing to try is how people end up accepting a confident answer because they are tired of asking.

Evaluation Is Where Verification Happens

Evaluation is the step where an invented detail is either caught or published. When you check a result, check the things that would matter if they were wrong — names, numbers, dates, quotes, claims about people, anything you would be embarrassed or liable to get wrong.

And be careful what you paste while iterating. Repeated attempts tempt people to supply more and more of the underlying material each round. The privacy rule from Module 3 does not relax because you are on your fourth try. Deeper prompt safety, privacy and prompt-injection awareness belong to Module 7.

Knowing When to Stop

Iteration has a cost, and beginners routinely overshoot: a fourth revision of an internal note is worse value than sending the second one. Ask "is this good enough for what it is actually for?" rather than "could this be better?" — the second question never terminates.

Two useful markers: stop when the remaining flaws are ones you would fix faster by hand than by re-prompting; and stop when the last two attempts differ in wording but not in usefulness.

Human + AI

You define what "good enough" means for this task, and you are the one who decides that the work is finished. An assistant can produce another version indefinitely; it has no view of whether the last one already served your purpose, and it cannot tell you that the problem is the task rather than the prompt.

• •An assistant will not tell you when it is wrong.
• •A better prompt does not fix every answer.
• •Confident does not mean correct.
• •Perfect is not the target; good enough for the actual purpose is.