Support Short Term SA Tokens
Bug
Description
Our current auto-init function no longer works due to k8s service account using short-term tokens.
If the vault pod is bounced, the registered k8s token that was installed on initial create is no longer valid, and the prometheus pod is not able to connect to vault.
We can fix this per Vault documentation by removing token_reviewer_jwt and kubernetes_ca_cert from our auth init commands. See https://developer.hashicorp.com/vault/docs/auth/kubernetes#kubernetes-1-21
This behavior can be demonstrated by:
- bounce the vault-vault-0 pod. A new service token is created under
/run/secrets/kubernetes.io/serviceaccount/token - bounce the
prometheus-monitoring-monitoring-kube-prometheus-0-- thevault-agent-initpod fails to connect, since vault's running service token has now changed from the initialvault write auth/kubernetes/config ...command
Edited by Michael Martin