Retry-with-Error-Feedback Pattern
CoreImplement validation, retry, and feedback loops for extraction quality · Difficulty 3/5
0%
validationretryerror-feedbackextraction
Prerequisites
When extraction or generation produces invalid output, a retry-with-error-feedback pattern guides the model toward correction by appending specific validation errors.
The Pattern
- The original source document
- The failed extraction/output
- Specific validation errors (not just "try again")
When Retries Work vs Fail
| Scenario | Retry Effective? | Why |
|---|
|----------|-----------------|-----|
| Format mismatch (date format wrong) | Yes | Model can reformat |
|---|---|---|
| Structural error (wrong nesting) | Yes | Model can restructure |
| Semantic error (values don't sum) | Yes | Model can recalculate |
| Information absent from source | No | Model will fabricate or hallucinate |
| Data in external document not provided | No | Model has no access to the information |
Key Distinction
Schema syntax errors are eliminated by tool use. Semantic validation errors (values don't sum to total, data in wrong fields) persist even with strict schemas and require validation + retry logic.
Self-Correction Validation Flows
calculated_total alongside stated_total to flag discrepanciesconflict_detected booleans for inconsistent source dataKey Takeaways
- ✓Append specific validation errors to the retry prompt -- not just 'try again'
- ✓Retries work for format/structural/semantic errors but fail when information is absent from the source
- ✓Tool use eliminates syntax errors; semantic errors require validation + retry logic
- ✓Extract computed validation fields (calculated_total, conflict_detected) for automated checking