Skip to content
AI

Why can I not reproduce the exact output my model gave in production?

85

Oportunidad

Even at temperature zero, the same prompt sent to the same model twice can return different tokens. A January 2026 arXiv paper traced the root cause to dynamic batching: GPU reduction kernels change their internal arithmetic tree when batch size changes, and floating-point addition is not associative, so two requests processed under different batch sizes take different numerical paths and diverge. Serving infrastructure has no obligation to record or expose the batch context that produced a given output. This makes it impossible to replay a failed inference, write a regression test that pins exact behavior, or reconstruct what a model actually said during a production incident. The only solution demonstrated so far imposes a 61 percent throughput cost.

Por qué importa

Deterministic replay is the minimum bar for treating a model call as an auditable computation rather than a black box you trust but cannot inspect.

Cómo evalúo la oportunidad

La Puntuación de Oportunidad es mi propia lectura, no una medición: cuánto duele, con qué frecuencia aparece y qué tan poco existe para resolverlo hoy. Un valor más alto significa que creo que vale más la pena construirlo.

Gravedad8/10

Cuánto dolor causa cuando aparece.

Frecuencia9/10

Con qué frecuencia la gente se topa con ello.

Espacio en blanco8/10

Qué tan pocas herramientas buenas existen para ello hoy.

Más problemas que vale la pena resolver