Good question with a few practical layers: ## Catching Hallucinations Mid-Stream ### 1. Structured output + schema validation Force the model into a typed schema (JSON mode, tool use, or `response_format`). If the output doesn't parse, reject before acting. ```python # Anthropic tool use — model *must* fill the schema response = client.messages.create( model="claude-sonnet-4-6", tools=[{"name": "action", "input_schema": your_schema}], tool_choice={"type": "tool", "name": "action"}, messages=[...] ) # If tool_use block is missing or malformed → hallucination signal ``` ### 2. Grounding assertions before execution After the model produces a plan, run a fast verification step *before* execution: ```python # Cheap, targeted check verifier_prompt = f""" Does this action reference real entities from the provided context? Context: {context} Proposed action: {action} Reply VALID or INVALID with one reason. """ ``` Use a cheap/fast model (Haiku) for the verifier — you don't need full reasoning power. ### 3. Confidence probing Ask the model to self-assess *as part of its output*, then gate on the score: ```python schema = { "answer": str, "confidence": 0–10, # forced field "sources_used": list[str] # forces citation } # Gate: if confidence < 7 or sources_used is empty → pause and ask user ``` ### 4. Streaming: token-budget interrupts If streaming, watch for divergence *before* the full response lands: - Use `max_tokens` budgets per step so a runaway generation fails cheap - With Claude's extended thinking, check `thinking` blocks for contradictions before the `text` block completes - Set up a secondary "sanity" prompt that runs on the first ~200 tokens if latency allows ### 5. Tool-call round-trips as a forcing function Structure your agent so it *must* call a lookup tool before acting on any factual claim. A hallucinated entity will fail the lookup, surfacing the problem before execution. ``` model → propose_action(entity_id="X") → lookup(entity_id="X") → 404 → reject, re-prompt with error ``` --- **The most reliable pattern** in practice: structured output + a cheap verifier + require explicit citations from the context window. The combination makes it structurally hard to act on a hallucination rather than relying on the model's self-awareness.