UNCLASSIFIED - NO CUI

Skip to content

Drift Correction Failed - gitlab-rails-secret-bb patch failure

Bug

Description

From PB Engineer on MM:

Describe the problem, what were you doing when you noticed the bug?

Troubleshooting a complaint that Flux has... We have drift detection enabled on our clusters. And it keeps choking on this:

  Warning  DriftCorrectionFailed  14m (x124 over 23h)  helm-controller  Failed to correct cluster state of release bigbang/bigbang.v8:                                                                                      
Secret/gitlab/gitlab-rails-secret-bb patch failure: illegal base64 data at input byte 0                                                                                                                                     

If we delete the secret it gets created juuuuust fine but something about the patch operation angers Flux. Looking at it, there's a newline / blank line at the top of the secret that I suspect is making some validator mad. In the BB chart, that secret is created with this as the data: {{ .Values.addons.gitlab.railsSecret | nindent 4 | b64enc }}

nindent is what's inserting that empty line. So the secret ends up looking like (I have removed Base64 encoding in this output to show the newline) :

data:
  secrets.yml: |
    
    somesecret

Anyone know why nindent is used so widely in the BB charts instead of just indent? Have you encountered this before / happen to have a workaround?

Also worth mentioning that at this point I'm just suspicious of the newline. Since the complaint is about byte 0 and the first character is a newline.... It's entirely possible I'm barking up the wrong tree, but I'm still curious why nindent is used.

Our workaround will probably be managing this outside of BB, which is always a crappy workaround.

This does appear to be the only place that combines nindent with b64enc.

Template Link: https://repo1.dso.mil/big-bang/bigbang/-/blob/master/chart/templates/gitlab/secret-rails.yaml?ref_type=heads#L10

BigBang Version

2.41.0