Dead-Letter Queues for LLM Extraction Failures: Capture, Triage, and Replay Without Losing Trust

A validation failure is not an exception to hide. It is a record your system does not yet know how to trust. That distinction matters in LLM extraction pipelines. A malformed invoice, an unexpected OCR layout, a model response that violates the schema, and a semantically impossible value may all reach the same line of validation code. If the only outcomes are “retry” or “drop,” the pipeline will either waste money repeating the same failure or silently lose work. The production answer is a dead-letter path: a durable place for failed records to wait with enough evidence to explain, triage, and safely replay them. The queue itself is the easy part. The hard part is designing the failure contract around it. Validation is where routing begins Constrained decoding and post-hoc validation solve different problems . Even with both, some records should fail. Real documents are messy, schemas change, OCR corrupts values, and models sometimes return plausible nonsense. A robust validation boundary should produce more than true or false . It should emit a reason the rest of the pipeline can act on: which schema and model versions were used which fields failed and why whether the payload was malformed, incomplete, or semantically invalid the confidence signal attached to the extraction whether a retry is likely to change the result That result becomes a routing decision. High-confidence, valid records can flow forward. Recoverable transport failures can use a bounded retry. Ambiguous or invalid records belong in review or a dead-letter queue. Confidence-based routing is useful precisely because “trust everything” and “review everything” are both bad operating models. A dead-letter record needs evidence, not just payload Putting the original input on another queue is not enough. Without context, the team investigating the failure has to reconstruct the run from scattered logs—if those logs still exist. I would store a dead-letter envelope containing: a stable record ID and idem