Design Prompts with Explicit Criteria
This module is about precision prompting: giving Claude specific reporting criteria so it flags real issues and avoids noisy, trust-damaging false positives.
The exam will likely test whether you can improve prompts like:
into concrete, categorical criteria like:
Report a comment issue only when the comment makes a specific behavioral claim that directly contradicts the current code behavior. Do not report vague, incomplete, outdated, stylistic, or non-behavioral comments.
Anthropic’s prompt engineering docs emphasize that clear, explicit instructions improve results, and that Claude performs better when you specify the desired output, constraints, and success criteria rather than relying on vague prompts. (Claude)
1. Core mental model
For review prompts, vague instructions produce noisy findings.
Vague prompt → Claude guesses what matters.
Explicit criteria → Claude knows what to report, skip, and how to classify severity.
The exam shortcut:
2. Why vague instructions fail
Weak prompt
Problem: “accurate” is too broad.
Claude might flag:
A comment that is slightly incomplete.
A comment that uses old terminology.
A comment that is stylistically vague.
A comment that could be clearer.
A comment that does not mention a new edge case.
Many of those are not review-worthy.
Better prompt
Flag comment issues only when all of these are true:
1. The comment makes a concrete claim about runtime behavior, data shape, security behavior, error handling, or side effects.
2. The current code directly contradicts that claim.
3. The contradiction could mislead a developer into using or modifying the code incorrectly.
Skip:
- comments that are merely incomplete
- comments that use older terminology but are not behaviorally wrong
- comments that are vague but not contradictory
- comments in generated files
- style or grammar issues
This prompt defines a category boundary.
Anthropic’s Code Review docs describe review customization in similar terms: you can tune what Claude flags, define severity, list skip rules, require evidence before findings, and suppress categories that are noisy or already covered by CI. (Claude)
3. “Be conservative” is weaker than explicit criteria
The official guide makes this point directly.
Weak:
These sound good, but they do not tell Claude which findings are allowed.
Better:
Report only these categories:
1. Correctness bugs introduced by this diff that can cause wrong output, crashes, data loss, or broken user-visible behavior.
2. Security issues introduced by this diff, including auth bypass, tenant isolation failure, sensitive data exposure, or injection risk.
3. Broken edge cases where the new code fails for null, empty, boundary, or common invalid inputs already handled elsewhere.
Do not report:
- formatting, lint, naming, or style issues
- issues CI already enforces
- purely hypothetical risks without a plausible execution path
- pre-existing issues not introduced by the diff
- minor local pattern differences with no behavioral impact
This converts “be conservative” into a clear decision rule.
4. Report vs skip criteria
A strong review prompt should say both:
Example: high-precision code review criteria
# Review Criteria
Report only issues introduced by this diff that meet one of these categories:
## Report
### Correctness
- The changed code can produce incorrect output for a realistic input.
- The changed code can throw an exception on a common path.
- The changed code breaks an existing documented behavior.
- The changed code mishandles null, empty, boundary, or invalid input in a way that affects runtime behavior.
### Security
- The changed code weakens authentication or authorization.
- The changed code exposes secrets, PII, tokens, or tenant data.
- The changed code introduces injection, path traversal, SSRF, XSS, or unsafe deserialization risk.
- The changed code logs sensitive request or user data.
### Data integrity
- The changed code can corrupt, drop, duplicate, or incorrectly migrate data.
- The changed code can create inconsistent state after retry or partial failure.
## Skip
Do not report:
- formatting, lint, or type errors already enforced by CI
- subjective style preferences
- minor naming differences
- pre-existing issues not introduced by this diff
- generated files, lockfiles, or vendored dependencies
- missing tests unless the missing test creates concrete regression risk under project policy
Claude Code’s Code Review docs say default reviews focus on correctness bugs that would break production, not formatting preferences or missing test coverage, and that review guidance can define skip rules such as generated files, lockfiles, vendored dependencies, and categories already enforced by CI. (Claude)
5. False positives damage developer trust
The official guide highlights an important human factor:
This is not just an accuracy problem. It is a workflow adoption problem.
If a review bot posts ten comments and seven are wrong or trivial, developers may start ignoring all comments, including the three real bugs.
Example
Claude reports:
Likely developer reaction:
Even the real tenant bugs lose credibility.
Exam implication
If one category has a high false-positive rate, temporarily disable or narrow that category.
Example:
Do not report performance findings for now unless there is direct evidence of an O(n²) operation on an unbounded production input path.
Skip speculative performance comments such as:
- "this might be slower"
- "consider caching"
- "could be inefficient"
Allowed performance finding:
- A changed loop now performs a database query once per user for an unbounded list, where the previous implementation batched the query.
This restores trust while you improve that category’s prompt.
6. Temporarily disabling noisy categories
The official skill requirement says you should know when to disable high false-positive categories.
Use this when:
A category is often wrong.
Developers are losing trust.
The prompt lacks enough criteria.
The category needs better examples or validation.
You need to preserve signal from accurate categories.
Bad response
Better response
Temporarily do not report comment accuracy findings.
Reason:
This category has produced too many false positives.
Replacement rule:
Only report comment/code mismatches when the comment makes a concrete behavioral claim that directly contradicts current code behavior and could mislead future maintainers.
Claude Code’s review customization docs explicitly support skip rules and higher bars for noisy categories or paths, such as “only report if near-certain and severe” for certain areas. (Claude)
7. Severity criteria
The exam specifically calls out defining severity criteria with concrete code examples.
A weak severity rubric:
This is too vague.
A better rubric:
# Severity Rubric
## Important
Report as Important only if the issue should be fixed before merging.
Important includes:
- production correctness bug
- security vulnerability
- data loss or corruption
- tenant isolation failure
- crash on a common user path
- broken documented behavior
- migration that is not idempotent when reruns are expected
## Nit
Report as Nit only if the issue is worth fixing but not merge-blocking.
Nit includes:
- small maintainability issue
- minor project convention violation
- unclear code that could reasonably confuse future maintainers
- missing low-risk test under project standards
## Skip
Do not report:
- pure style preferences
- formatting/lint/type issues already handled by CI
- speculative concerns without evidence
- pre-existing issues
- generated files
Claude Code’s Code Review docs define severity categories such as Important, Nit, and Pre-existing, and note that review guidance can redefine what Important means for a repository. (Claude)
8. Severity examples with code
Concrete examples make the severity rubric more consistent. Anthropic’s prompting docs say examples are one of the most reliable ways to steer Claude’s output format, tone, and structure, and recommend relevant, diverse, structured examples. (Claude)
Example 1: Important correctness bug
// New code
function getDiscountPercent(user: User) {
if (user.plan = "enterprise") {
return 20;
}
return 0;
}
Expected classification:
Why:
Assignment instead of comparison changes runtime behavior and grants the enterprise discount incorrectly.
Example 2: Important security issue
// New code
app.get("/admin/users", async (req, res) => {
const users = await db.users.findMany();
res.json(users);
});
Expected classification:
Why:
Example 3: Nit
Expected classification:
Why:
Example 4: Skip
Potential finding:
Expected classification:
Why:
9. Evidence requirements
One of the best ways to reduce false positives is to require evidence.
Weak:
Better:
For every finding, include:
1. The exact changed line or block.
2. The execution path that triggers the issue.
3. The expected behavior.
4. The actual behavior caused by the diff.
5. Why this was introduced by the diff, not pre-existing.
6. A minimal example input if applicable.
Do not report if you cannot provide this evidence.
Claude Code’s review docs recommend requiring evidence for classes of findings; for example, behavior-claim findings should cite source evidence rather than infer from naming. (Claude)
10. Example: comment accuracy prompt
Bad prompt
Exam-ready prompt
# Comment Accuracy Review
Report comment issues only when all conditions are met:
1. The comment makes a specific behavioral claim about what the code does.
2. The current implementation directly contradicts that claim.
3. The contradiction could mislead a future developer about runtime behavior, data shape, side effects, error handling, security behavior, or performance characteristics.
4. You can cite the comment line and the contradictory code line.
Do not report:
- comments that are merely incomplete
- comments that are outdated only in terminology
- comments with grammar/style issues
- comments that omit edge cases but do not contradict the code
- comments in generated files
- comments that are obviously examples or TODOs
Severity:
- Important: misleading comment could cause incorrect API use, security misunderstanding, data-loss risk, or wrong production behavior.
- Nit: misleading but low-risk maintainability issue.
- Skip: style, grammar, vague comments, or incomplete but not contradictory comments.
11. Example: review prompt with explicit categories
You are reviewing a pull request diff.
Only report issues introduced by this diff.
## Report Categories
### Important
Report as Important only if the issue should block merge:
- correctness bug on a realistic production path
- security vulnerability
- tenant isolation failure
- data loss, corruption, duplicate writes, or non-idempotent migration
- crash or unhandled exception on common valid input
- broken documented behavior
### Nit
Report as Nit only if worth fixing but not merge-blocking:
- small maintainability issue likely to confuse future maintainers
- minor project convention violation not covered by CI
- missing test for a meaningful but low-risk branch
## Skip Categories
Do not report:
- formatting, lint, type errors, or import sorting
- style preferences
- speculative performance concerns without evidence
- missing tests for trivial branches
- generated files, lockfiles, or vendored code
- pre-existing issues
- code outside the diff unless necessary to explain the bug
## Evidence Requirement
For every finding, provide:
- file and line
- severity
- category
- exact evidence from code
- realistic failure scenario
- suggested fix
Do not report if you cannot provide a realistic failure scenario.
12. How to tune a noisy review category
Suppose your review prompt currently includes:
Claude posts many weak findings:
Step 1: Disable or narrow temporarily
Temporarily skip performance findings unless there is direct evidence of a newly introduced unbounded operation on a production path.
Step 2: Add categorical criteria
Report performance issues only when:
1. The diff introduces a repeated network call, database query, or expensive computation inside a loop over an unbounded collection.
2. The previous code avoided that repeated operation.
3. The input size can realistically grow in production.
4. The finding includes the loop, the repeated operation, and the production path.
Step 3: Add examples
Report:
for (const user of users) {
await db.orders.findMany({ where: { userId: user.id } });
}
Reason:
This introduces one query per user for an unbounded production list.
Skip:
const displayNames = users.map(user => formatName(user));
Reason:
This is an in-memory linear transformation with no evidence of production risk.
13. Prompt patterns to memorize
Pattern A: report/skip criteria
Pattern B: evidence gate
Do not report unless you can show:
- changed code line
- realistic failure path
- expected vs actual behavior
- why it was introduced by this diff
Pattern C: severity rubric
Important = must fix before merge because...
Nit = worth fixing but not blocking because...
Skip = not worth reporting because...
Pattern D: temporary category disablement
Temporarily do not report [category] findings.
Only re-enable when the prompt defines exact reportable conditions and examples.
Pattern E: examples per severity
Here are examples of Important, Nit, and Skip findings. Classify new findings consistently with these examples.
14. Scenario-based practice questions
Question 1
A review prompt says:
Claude reports many comments that are merely incomplete or stylistically vague.
What is the best prompt improvement?
A) Repeat “be conservative” three times.
B) Define reportable comment issues as concrete behavioral contradictions between comment and code, and list incomplete/style comments as skip cases.
C) Ask Claude to use fewer words.
D) Disable all code review.
Answer
B. Specific categorical criteria outperform vague confidence-based instructions. The prompt should define exactly when a comment issue is reportable.
Question 2
A team says “only report high-confidence findings,” but Claude still reports speculative performance concerns.
What is the best fix?
A) Use a higher temperature.
B) Define exact performance-reporting criteria, such as repeated database calls inside unbounded production loops, and skip speculative “consider caching” comments.
C) Ask Claude to be nicer.
D) Report all performance issues as Important.
Answer
B. “High confidence” is not specific enough. The prompt must define the category boundary.
Question 3
A review bot has accurate security findings but noisy style and comment findings. Developers begin ignoring the bot entirely.
What should the team do?
A) Temporarily disable or narrow the high-false-positive categories while preserving accurate categories.
B) Keep all categories and ask developers to read carefully.
C) Make every finding Important.
D) Remove severity labels.
Answer
A. Noisy categories damage trust in the entire system. Disable or narrow those categories until the prompt has better criteria.
Question 4
Which is the best severity definition for Important?
A) “Anything that seems serious.”
B) “Any issue that should block merge, such as a production correctness bug, security vulnerability, data corruption risk, tenant isolation failure, or crash on a common path.”
C) “Anything Claude is confident about.”
D) “Any style issue.”
Answer
B. Severity should be tied to concrete impact and merge-blocking criteria, not vague seriousness or confidence.
Question 5
A prompt says:
Claude reports import ordering and formatting comments that CI already enforces.
What should the prompt say?
A) “Report issues more carefully.”
B) “Do not report formatting, lint, type, import ordering, or style issues already enforced by CI.”
C) “Ignore CI.”
D) “Report all nits as Important.”
Answer
B. Skip criteria should explicitly exclude categories covered by CI.
Question 6
A review finding claims a behavior bug, but Claude cannot provide an execution path or expected-vs-actual behavior.
What should the prompt require?
A) Report it anyway.
B) Do not report unless the finding includes code evidence, a realistic execution path, and expected vs actual behavior.
C) Convert it to Important.
D) Ask the developer to investigate.
Answer
B. Evidence gates reduce false positives by requiring a concrete failure scenario.
Question 7
A docs repository gets review findings calibrated for production code severity. Many findings are too aggressive.
What should the team customize?
A) The severity rubric, especially what counts as Important for this repository.
B) The file extension.
C) The model name only.
D) The git branch name.
Answer
A. Severity criteria should match repository context. A docs repo may need a narrower definition of Important than a production service.
Question 8
A prompt currently flags missing tests for nearly every change, including trivial copy changes.
What is the best improvement?
A) “Be conservative about tests.”
B) “Report missing tests only when the diff changes behavior, introduces a bug fix, changes business logic, or adds an API path without required integration coverage. Skip copy-only or refactor-only changes with no behavior change.”
C) “Always report missing tests.”
D) “Never report missing tests.”
Answer
B. Explicit criteria define when missing-test findings are reportable.
Question 9
A team wants consistent severity classification.
What should they add?
A) Concrete examples for Important, Nit, and Skip findings.
B) A vague definition of “serious.”
C) More categories with no definitions.
D) Only a confidence threshold.
Answer
A. Examples help Claude classify consistently and reduce ambiguity.
Question 10
A review category has repeated false positives, but the team believes the category is valuable if tuned correctly.
What is the best short-term response?
A) Keep posting all findings while improving later.
B) Temporarily disable or narrow the category, then re-enable after adding explicit criteria and examples.
C) Delete all review prompts.
D) Mark all findings as Nit.
Answer
B. Temporarily disabling a noisy category protects trust while the prompt is improved.
15. Final D4.1 checklist
Memorize this:
1. Explicit criteria beat vague instructions.
2. “Be conservative” is weaker than report/skip rules.
3. “Only high-confidence findings” is weaker than categorical criteria.
4. Define what to report and what to skip.
5. Focus review prompts on bugs, security, regressions, and concrete behavior.
6. Skip style, formatting, lint, and issues CI already catches.
7. Require evidence for findings.
8. Require realistic failure scenarios.
9. False positives reduce developer trust.
10. Noisy categories can damage trust in accurate categories.
11. Temporarily disable high-false-positive categories while improving prompts.
12. Define severity levels with concrete impact criteria.
13. Important should mean “must fix before merge.”
14. Nit should mean “worth fixing but not blocking.”
15. Add concrete code examples for Important, Nit, and Skip.
D4.1 mental model
Bad prompt: “Find issues. Be conservative.”
Good prompt: “Report only these issue categories, skip these categories, require this evidence, and classify severity using these concrete examples.”
The exam’s favorite distinction is:
Confidence-based wording does not reliably reduce false positives. Explicit report/skip criteria and severity examples do.