Leaderboard scores collapse for a few interconnected reasons: ## Benchmark Contamination Most popular benchmarks (MMLU, HumanEval, GSM8K, etc.) have been on the internet for years. Training corpora scraped from the web almost certainly contain the questions, answers, or near-duplicates. The model isn't reasoning -- it's pattern-matching against memorized answers. ## The Overfitting Hierarchy There are three layers of overfitting happening simultaneously: 1. **Direct memorization** -- the model has seen the exact question-answer pair 2. **Distribution leakage** -- the model has seen enough similar examples that it has learned the benchmark's specific style, format, and answer patterns rather than the underlying skill 3. **Meta-overfitting** -- the broader research community iterates on architectures/training recipes *using benchmark scores as the optimization target*, so the field as a whole overfits to the benchmark ## Why Fresh Test Sets Expose This When you evaluate on a genuinely unseen test set: - Memorized answers no longer apply - Benchmark-specific formatting tricks don't transfer - The score reverts toward the model's true generalization capability The gap between the contaminated score and the clean score is roughly the measure of how much the score was "borrowed" from training data rather than earned through generalization. ## The Deeper Issue Generalization is hard. A model trained to predict text tokens has no obligation to learn the abstract reasoning structure behind a problem -- it only needs to minimize loss on its training distribution. If the benchmark is *in* that distribution, it will look capable. On a shifted distribution, the shortcut breaks. This is why papers like **HELM** and **BIG-Bench Hard** tried to use novel or held-out tasks, and why dynamic benchmarks (where questions are generated fresh each eval) are gaining traction -- they close the gap between "has seen this" and "can actually do this."