Kubernetes 1.37 turns native histograms on by default for its own metrics

Kubernetes 1.37 flips native histograms on by default for its own metrics, per the release blog. It is a beta graduation of the change that landed in 1.36 alpha under KEP-5808, and it targets one specific pain: the shape of the duration data the control plane exposes to Prometheus.

Enabled by default means most clusters upgrading to 1.37 will start emitting the newer format without an opt-in flag. Whether your monitoring stack ingests it is a separate question.

Classic histograms, briefly

A classic Prometheus histogram exports one time series per bucket boundary. A ten-bucket histogram with a handful of labels quietly turns into dozens of series, and the TSDB pays for every one of them. Get the boundaries wrong at design time and the data is quietly useless: latency that shifts into microseconds, or blows past the top bucket, vanishes into +Inf, and any histogram_quantile call over coarse buckets interpolates its way to a plausible-looking wrong answer. The Kubernetes blog names all three of these: the bucket guessing game, cardinality cost, and interpolation error.

What ships in 1.37

Native histograms replace the fixed-bucket exposition with dynamically-sized buckets driven by the observed distribution. The blog cites duration and latency metrics as the target, calling out API server request latencies and scheduling durations. The KEP has been baking since 1.36; the beta step means enabled by default, still tunable through the standard feature-gate machinery if you need to back it out.

The release calls the result high-resolution, low-cardinality observability, with a meaningful drop in telemetry storage and scraping overhead. That is the design goal restated, and the mechanism plausibly delivers it. Nothing in the release notes says otherwise.

Where a CI/CD team feels it

Anywhere a pipeline reads control-plane latency percentiles to gate a step. If your progressive-delivery job queries API server request-duration percentiles to decide "cluster is healthy, promote the canary," the shape of the data behind that query is different in 1.37. Higher-resolution percentiles are the upside. The downside is that any alert threshold calibrated against interpolated numbers from classic buckets may fire at slightly different times, or stop firing, once the histogram gets more honest about what actually happened.

Same story for the internal SLO dashboards that platform teams point at their clusters. The number you looked at last week and the number you look at next week are computed by different code paths, even if the label set on the graph is identical.

The scrape side is the catch

Native histograms are a Prometheus feature. Your scrape stack has to understand them to store them. If you run an older Prometheus, an ingester that only speaks classic exposition, or a hosted metrics vendor that has not shipped native-histogram support yet, the new series can be dropped or downgraded to the classic representation on the wire, and the storage and quantile benefits go with them. The Kubernetes post does not enumerate a minimum stack; check your ingester's own release notes before you count the win. OpenTelemetry's exponential histogram is a sibling design solving the same problem from a different direction, so if your pipeline already routes through an OTel collector there is a translation step to think through as well.

Second catch: it is beta. Beta in Kubernetes has meant "enabled by default and generally reliable" for a while now, but it is still not GA. If the platform team has a rule about only running GA features on production clusters, native histograms sit on the wrong side of that line until a later release lifts them.

Rollback path

Standard feature-gate disable across the API server, controller manager and scheduler, followed by a restart. That is the operational read: nothing you have not done before, no data migration to unwind, dashboards revert to their previous shape the moment the classic exposition comes back. Worth walking through on a lab cluster before the upgrade, not during it.