fix(metrics): preserve metrics user password across renders
Summary
metricsUserSecret.yaml regenerates TWISTLOCK_METRICS_PASSWORD with randAlphaNum on every Helm render, and only the init Job reconciles the new value into the Twistlock bigbang-metrics-sa user account. twistlock-init.sh runs set -e and dispatches metrics after license, DNS, SSO, users, and defenders, so any earlier failure leaves the Secret holding a password the user account has never seen. Prometheus then 401s on every scrape.
Two changes in one MR:
- Add a
lookupfallback inmetricsUserSecret.yamlso the password is generated once on first install and preserved on every subsequent render. Stops the drift and stops gratuitous password rotation on chart bumps. Matches the pattern used by other BB packages for persistent chart-managed secrets. - Move the metrics dispatch in
twistlock-init.shto run directly after license. Metrics only needs the auth token and license. It doesn't need DNS, SSO, users, or defender rollout. Narrows the remaining failure window if the password ever does need to be rewritten.
No changes to values, Secret name, Secret keys, the ServiceMonitor, or twistlock-metrics.sh itself. Chart bumped 0.25.0-bb.0 -> 0.25.0-bb.2 (reserving -bb.1 for the in-flight console.extraEnv MR so the two don't collide on the same fork-patch).
Drive-by: stripped stray trailing tabs in the Groups block of twistlock-init.sh while touching the file.
Relevant logs/screenshots
New helm-unittest suite (chart/tests/metrics_secret_test.yaml):
$ helm unittest chart -f 'tests/metrics_secret_test.yaml'
### Chart [ twistlock ] chart
PASS test metrics user secret persistence chart/tests/metrics_secret_test.yaml
Charts: 1 passed, 1 total
Test Suites: 1 passed, 1 total
Tests: 6 passed, 6 totalCovers: conditional render gates (monitoring.enabled, init.enabled), Secret name and kind, TWISTLOCK_METRICS_USER pinned to bigbang-metrics-sa, password is non-empty and base64 charset on fresh render, app.kubernetes.io/component: monitoring label present.
Shellcheck clean on the reordered init script:
$ bash -n chart/scripts/twistlock-init.sh && shellcheck chart/scripts/twistlock-init.shRendered output still quotes the password correctly:
$ helm template t chart/ --set init.enabled=true --set monitoring.enabled=true \
--show-only templates/init/metrics/metricsUserSecret.yaml | tail -3
stringData:
TWISTLOCK_METRICS_USER: "bigbang-metrics-sa"
TWISTLOCK_METRICS_PASSWORD: "dFZzdmw3ckhyNklUc2lyVXU4bDdWUDloTjJZSUlyVnI="Idempotency on upgrade (run against a dev cluster with the chart installed, identical --set flags on both invocations so the only variable is the lookup path):
PW1=$(kubectl get secret -n twistlock twistlock-console-metrics-auth \
-o jsonpath='{.data.TWISTLOCK_METRICS_PASSWORD}' | base64 -d)
helm upgrade t chart/ --set init.enabled=true --set monitoring.enabled=true
PW2=$(kubectl get secret -n twistlock twistlock-console-metrics-auth \
-o jsonpath='{.data.TWISTLOCK_METRICS_PASSWORD}' | base64 -d)
[ "$PW1" = "$PW2" ] && echo PASS || echo FAIL
# PASSRecovery (delete Secret, confirm regeneration + init Job resyncs the user):
kubectl delete secret -n twistlock twistlock-console-metrics-auth
helm upgrade t chart/ --set init.enabled=true --set monitoring.enabled=true
kubectl logs -n twistlock -l job-name=twistlock-init-<timestamp> \
| grep "metrics service account user 'bigbang-metrics-sa'"
# Ensuring default metrics service account user 'bigbang-metrics-sa' ... OKLinked Issue
Upgrade Notices
N/A