UNCLASSIFIED - NO CUI

Gateway Pods Crashing on RKE2

In our RKE2 pipelines, the loki gateway pod is stuck in a CrashLoopBackOff:

logging                    logging-loki-gateway-5b9f54d668-brqt8                                  1/2     CrashLoopBackOff   10 (87s ago)    28m

From the logs, it appears that it is referencing the wrong dnsService for RKE2:

[pod/logging-loki-gateway-5b9f54d668-brqt8/nginx] 2024/06/10 14:13:41 [emerg] 1#1: host not found in resolver "kube-dns.kube-system.svc.cluster.local." in /etc/nginx/nginx.conf:33
[pod/logging-loki-gateway-5b9f54d668-brqt8/nginx] nginx: [emerg] host not found in resolver "kube-dns.kube-system.svc.cluster.local." in /etc/nginx/nginx.conf:33

This is configurable via the helm chart, and I am testing fixes in this MR. Opening this issue so that we can track this as a required change/upgrade notice for RKE2 users.

Edited by Noah Birrer