Why do I only find out what dangerous things my model can do after it ships?
機会
Automated red-teaming tools such as GCG, AutoDAN, and PAIR report attack success rates of 5 to 15 percent and give a false sense of safety. Multi-turn human red-teaming on the same models finds failures up to 75 percent of the time on the same categories. The gap means that dangerous capability uplift in areas like bioweapon synthesis guidance or offensive cyber is being missed at the automated pre-deployment stage and found in the field instead. Frontier labs run their own manual evaluations under frameworks like Anthropic's RSP and METR's TaskDev, but the methodology is undocumented and non-standardized enough that no two labs run comparable tests. NIST's AI agent red-teaming guidance was still an annotated outline in early 2026, with full publication expected late 2026 to 2027.
重要な理由
A dangerous emergent capability discovered post-deployment in a widely distributed model is a different order of problem than one caught before release.
機会をどう評価するか
Opportunity Scoreは測定値ではなく、私自身の見解です。どれほど痛みを伴うか、どれほど頻繁に影響を与えるか、そして今日時点で解決策がいかに少ないか。スコアが高いほど、構築する価値が高いと私は考えています。
それが現れたときにどれほどの痛みをもたらすか。
実際にどれほど頻繁に人々がそれに直面するか。
今日時点で、それに対する優れたツールがいかに少ないか。