Why can a neighbor on my inference cluster reconstruct what I just asked the model?
机会
Modern LLM serving frameworks like vLLM share KV-cache blocks across requests with matching prefixes to reduce compute cost. When multiple tenants share the same inference server, an adversary tenant can probe cache-hit and cache-miss latencies to reconstruct another tenant's private prompt. Three independent attacks published in 2025 and 2026, PROMPTPEEK, EarlyBird, and InputSnatch, demonstrate up to 100% prompt reconstruction accuracy against production-grade serving stacks. Providers can defeat this by running fully isolated instances per tenant, but that eliminates all the memory and compute savings that make shared inference economically viable. The KVGov governance paper from August 2026 proposes per-principal cryptographic cache salts as a fix, but no commercial inference provider has deployed it.
为什么重要
Shared inference is the cost model that makes AI accessible at scale, but right now it cannot safely carry any sensitive workload without full tenant isolation.
我如何评估机会
机会评分是我的个人判断，而非量化指标：痛苦程度、发生频率，以及当前解决方案的匮乏程度。分数越高，意味着我认为越值得去构建。
出现时造成的痛苦程度。
人们实际遇到它的频率。
当前针对它的优质工具有多匮乏。