Why can I not trust a model's confidence score when it matters most?

Modern language models routinely output high-confidence tokens on wrong answers and low-confidence tokens on correct ones. The gap between stated probability and actual accuracy, called calibration error, has been documented across frontier models in a 2025 survey covering entropy, logit, and perturbation based methods. Production agents that use these scores to decide when to defer or abstain inherit the miscalibration directly, so they either hallucinate forward with false certainty or refuse correct answers unnecessarily. No off-the-shelf primitive gives a calibrated, actionable uncertainty signal cheap enough to run at inference time on every output token in a streaming response.