Who audits the experiment when the experimenter is the model itself?
機会
Autonomous AI systems are being deployed across biology, materials science, and software engineering to design experiments, run them, and write findings with minimal human review per step. A 2026 survey of AI scientist systems found that only 38 percent release the seeds or execution traces needed for a reviewer to reproduce a run, and exactly one of nine fully autonomous systems has been externally validated. There is no mechanism to verify that the steps a model reports match what was actually executed, or that its statistical summaries correctly reflect the underlying data. Human spot-checking does not scale to the volume these systems now produce.
重要な理由
Scientific findings compound through citation, so one convincing but unverifiable AI result can corrupt downstream research faster than any correction process can catch.
機会をどう評価するか
Opportunity Scoreは測定値ではなく、私自身の見解です。どれほど痛みを伴うか、どれほど頻繁に影響を与えるか、そして今日時点で解決策がいかに少ないか。スコアが高いほど、構築する価値が高いと私は考えています。
それが現れたときにどれほどの痛みをもたらすか。
実際にどれほど頻繁に人々がそれに直面するか。
今日時点で、それに対する優れたツールがいかに少ないか。