4.1 Prompts with Explicit Criteria

Domain 4 is about the craft of the prompt itself — getting Claude to produce precise, reliable output. Task Statement 4.1 starts with the most common prompting mistake: vague instructions. You tell Claude to 'be conservative' or 'only report high-confidence issues,' and you get inconsistent results — sometimes too many false alarms, sometimes missed problems. Why?

Because instructions like 'be careful' or 'be conservative' give Claude no actual DECISION BOUNDARY. Imagine training a new reviewer by telling them only 'flag the important stuff.' Important by whose definition? They'll guess, and guess differently each time. Now imagine instead handing them a checklist: 'Flag a comment ONLY when the claimed behavior contradicts what the code actually does. Do NOT flag style preferences.' That's a crisp, verifiable rule — anyone applying it lands in the same place. The difference between vague and explicit instructions is the difference between guessing and following a rule.

This is the core of 4.1: replace vague qualitative instructions with EXPLICIT CATEGORICAL criteria — concrete statements of exactly what to flag and what to skip. Explicit criteria are how you make Claude's judgment consistent and verifiable. The lesson also covers a counterintuitive trick for rescuing trust when one category goes wrong, and why 'just be more confident' isn't the fix people think it is.

An explicit criterion names the category and the precise condition. Instead of 'check that comments are accurate' (vague), write 'flag a comment only when the claimed behavior contradicts the actual code behavior' (explicit). The second version is verifiable — you can look at any comment and definitively say whether the rule applies. That verifiability is what makes the output consistent across runs and across cases.

The same applies to deciding WHAT to report at all. Rather than 'report important issues,' specify the categories to report (bugs, security vulnerabilities) versus the categories to skip (minor style nits, local stylistic patterns). And for severity, don't describe levels in prose ('high severity means serious') — show actual CODE PATTERNS for each level, so 'high' has a concrete, consistent meaning. Explicit criteria turn fuzzy judgment calls into rule-following.

Here's a subtle dynamic the exam tests. Suppose your review system has several categories, and ONE of them — say 'documentation mismatch' — produces a lot of false positives (40% of its flags are wrong). What's the damage? It's not just that one noisy category. Once developers see that category crying wolf, they start ignoring ALL the system's findings, including the accurate ones. A high false-positive rate in one category erodes trust in EVERY category.

The fix is counterintuitive, and that's exactly why it's tested. You might think 'tighten that category's confidence threshold.' Instead, the right move is to temporarily DISABLE the problematic category entirely while you refine its criteria with explicit examples — then re-enable it once it's reliable. Pulling the noisy category restores developers' trust in the rest immediately, and you fix the bad category offline rather than letting it poison the whole system in production. Trust is the asset you're protecting; a single noisy category can bankrupt it.

A tempting shortcut for precision is to have the model rate its own confidence and filter out low-confidence findings. The exam wants you to know why this is a weak primary strategy: model self-confidence is POORLY CALIBRATED. The model is often confidently wrong and hesitantly right, so its confidence score isn't a reliable proxy for correctness. Filtering on it lets confident errors through and discards uncertain-but-correct findings.

So the ordering matters: get the EXPLICIT CRITERIA right first — that's what actually improves precision — and treat confidence-based routing as a secondary refinement, not the main lever. Whenever an exam answer offers 'add a confidence threshold' or 'tell it to be more confident' as the fix for inconsistent precision, be suspicious: the real fix is almost always sharper criteria (and, when a category is noisy, disabling and refining it).

The 4.1 traps revolve around reaching for vague instructions or confidence thresholds where explicit criteria are the real fix. The signature scenario: a review category has a 40% false-positive rate; what do you do?

• •Vague instruction as a fix. ✗ 'Tell it to be more conservative' to reduce false positives. ✓ Write explicit categorical criteria for what to flag vs skip.
• •Confidence threshold as the cure. ✗ 'Raise the confidence threshold' for a noisy category. ✓ Disable the category, refine its criteria with examples, re-enable.
• •Keeping a noisy category live. ✗ Leaving the 40%-FP category active while you tweak it. ✓ Disable it to restore trust in the others, fix it offline.
• •Trusting self-confidence. ✗ Filtering findings purely on the model's self-rated confidence. ✓ Explicit criteria first; confidence only as a calibrated secondary signal.

You now know why vague instructions fail, how to write explicit criteria, the false-positive trust dynamic, and why confidence filtering is secondary. The exercise has you turn a vague review prompt into a precise one and feel the consistency improve.

Explicit criteria make judgment consistent through clear rules. The next lesson, 4.2, covers the most powerful technique when even explicit instructions aren't enough: few-shot examples — showing Claude what you want.