UNCLASSIFIED - NO CUI

Fix Mimir OTEL metric ingestion

General MR

Summary

Updates Mimir and Monitoring configuration to support OTEL-formatted metrics with UTF-8 metric and label names.

  • Configures Mimir with limits.name_validation_scheme: utf8 so OTEL-formatted labels such as http.method are accepted.
  • Removes the temporary Prometheus writeRelabelConfigs workaround that dropped metric names containing dots and stripped labels containing dots before remote write to Mimir.
  • Adds unit test coverage to verify the Mimir UTF-8 name validation configuration is rendered.
  • Verified through Helm rendering that name_validation_scheme: utf8 is present and the temporary remote write filtering is no longer rendered.

Relevant logs/screenshots

Screenshot 2026-08-24 at 2.23.38 PM.png

Screenshot 2026-08-24 at 2.24.30 PM.png

Helm render confirms limits.name_validation_scheme: utf8.

Debug Mimir distributor logs show no err-mimir-label-invalid / HTTP 400 invalid-label errors while Prometheus remote-write traffic was reaching Mimir.

Linked Issue

big-bang/product/packages/mimir#80

Upgrade Notices

Mimir now accepts UTF-8 metric and label names, and Monitoring no longer drops names containing dots before remote write. Environments receiving OTEL metrics may observe increased ingested series, cardinality, and storage usage. Review Mimir capacity and retention settings if necessary; no action is otherwise required.

Edited by Jeffrey Victor

Merge request reports

Loading