UNCLASSIFIED - NO CUI

Skip to content

BBv1.15.0 upgrade from BBv1.14.1

Bug

Description

Upgrading from BBv1.14.1 to BBv1.15.0, monitoring wouldn't reconcile here is what I found: kubectl get events -A|grep gatekeeper.sh monitoring 4m6s Warning FailedCreate daemonset/monitoring-monitoring-prometheus-node-exporter (combined from similar events): Error creating: admission webhook "validation.gatekeeper.sh" denied the request: [host-networking] The hostPort 9100 is not allowed in container node-exporter. Allowed: 0 <= hostPort <= 0

flux get hr -A bigbang monitoring False Helm rollback failed: release monitoring-monitoring failed: timed out waiting for the condition 14.0.0-bb.3 False

                                    Last Helm logs:                                                                                                                                      
                                    Looks like there are no changes for Service "monitoring-monitoring-grafana"                                                                          
                                    Looks like there are no changes for Service "monitoring-monitoring-kube-state-metrics"                                                               
                                    Looks like there are no changes for Service "monitoring-monitoring-prometheus-node-exporter"                                                         
                                    beginning wait for 134 resources with timeout of 10m0s                                                                                               
                                    DaemonSet is not ready: monitoring/monitoring-monitoring-prometheus-node-exporter. 6 out of 7 expected pods have been scheduled 

I fixed it with adding gatekeeper override to exclude monitoring-monitoring-prometheus-node-exporter for hostNetworking.