Fix Loki connectivity and tenant authentication for Alloy and Grafana
Package Merge Request
Package Changes
This MR updates the Big Bang umbrella chart templates to resolve Loki log ingestion and Grafana datasource connectivity issues when deploying Loki with the scalable strategy.
The changes ensure Alloy can reliably push logs to Loki through the gateway, authenticate using the configured Loki tenant, and allow Grafana to query logs using the same tenant.
Changes
- Alloy Loki destination and connectivity
Update the Loki service mapping to route scalable and distributed deployments through logging-loki-gateway.
Update the Loki push endpoint to use the gateway service port (80).
Add the X-Scope-OrgID tenant header to Alloy's Loki write destination to prevent HTTP 401 Unauthorized responses.
Update Alloy egress NetworkPolicies to allow traffic to the Loki gateway pod on TCP port 8080, which is the gateway's target port.
Apply the egress policy updates to both Alloy logs and Alloy singleton workloads.
- Loki ingress NetworkPolicies
Allow Alloy logs and Alloy singleton workloads to reach the Loki gateway in scalable and distributed deployments.
Preserve the existing Grafana gateway access and monolith ingress behavior.
Scope gateway access using the Loki gateway pod labels.
- Grafana Loki datasource
Configure the Loki datasource to send the X-Scope-OrgID header.
Ensure the tenant header is configured whenever Loki is enabled, independently of whether Tempo is enabled.
Preserve Tempo trace-to-logs and derived-field integration when Tempo is enabled.
Problem addressed
The original configuration caused two connectivity issues:
Alloy was targeting the Loki write service directly rather than the gateway endpoint used by the scalable deployment. The associated NetworkPolicy also allowed the service port instead of the gateway pod's target port.
Loki multi-tenancy requires an organization ID. Alloy and Grafana requests without the matching X-Scope-OrgID header resulted in unauthorized requests or queries that returned no logs.
Package MR
For Issue
Closes (link to issue here)
big-bang/product/packages/loki#329 (closed)
Upgrade Notices
N/A