UNCLASSIFIED - NO CUI

Skip to content

Resolve "Allow alloy to connect to prometheis, loki, and tempo to push data"

Zach Callahan requested to merge 38-allow-downstream-telemetry-ingest into main

Alloy Network Policies

Summary

This MR narrows the network policies for Alloy, allowing egress to the k8s API, tempo, loki, and prometheus. I also implemented a pattern that allows selectors and/or ports to be overridden per downstream service so please take a look at that.

Alloy is still struggling with pushing metrics but that's due to issues with the endpoint and not connectivity. This is visible in the logs below. Debugging further would likely be out of scope so I'm gonna save that for another ticket.

Relevant logs/screenshots

alloy ts=2024-09-11T19:35:53.705965331Z level=error msg="non-recoverable error" component_path=/ component_id=prometheus.remote_write.metrics_service subcomponent=rw remote_name=2cb11b url=http://monitoring-monitoring-kube-prometheus.monitoring.svc.cluster.local:9090/api/prom/push count=1380 exemplarCount=0 err="server returned HTTP status 404 Not Found: 404 page not found"

Upgrade Notices

N/A

Closes #38 (closed)

Edited by Zach Callahan

Merge request reports