Skip to content
Tech

Why do I only find out I tagged a metric wrong when the invoice arrives?

82

Kans

Tagging a metric with a high-cardinality attribute like a user ID, session ID, or request path multiplies a single time-series into hundreds of millions, and the bill follows a week later rather than a CI failure follows a minute later. By 2026, observability spend has reached 15 to 25 percent of total cloud spend at affected SaaS companies, with documented single-service incidents generating $90,000 in charges over a weekend. Every major backend, including Datadog, Grafana Cloud, and Prometheus, detects cardinality explosions only after ingestion, meaning the only signal is a budget alert or an invoice line item, not a pull request annotation. The OpenTelemetry SDK added optional cardinality caps in early 2026, but these are runtime circuit-breakers that silently drop data rather than upstream gates that catch the problem before it ships. There is no widely adopted CI primitive that ana

Waarom het belangrijk is

Every engineer who has tagged a metric with a user ID has had a bad Tuesday; catching it at the pull request instead of on the invoice permanently changes the economics of instrumentation.

Hoe ik de kans beoordeel

De Opportunity Score is mijn eigen inschatting, geen meting: hoe ernstig het pijn doet, hoe vaak het toeslaat, en hoe weinig er vandaag de dag bestaat om het op te lossen. Hoger betekent dat ik denk dat het meer de moeite waard is om te bouwen.

Ernst8/10

Hoeveel pijn het veroorzaakt wanneer het zich voordoet.

Frequentie8/10

Hoe vaak mensen er daadwerkelijk tegenaan lopen.

Witruimte8/10

Hoe weinig goede tools er vandaag de dag voor bestaan.

Meer problemen die het oplossen waard zijn