Bug: Flux Install script was updated to leverage a broken/buggy IronBank Kustomization Controller

Bug

Description

Describe the problem, what were you doing when you noticed the bug?
While, doing a dry running through the BB Engineering Cohort labs I noticed the labs randomly broke, spent hours figuring out went wrong.

If a Mac User with gpg version 2.2.27 (Long Term Support Edition, which has worked in the past, and the 2.3.x latest version of gpg tends to be broken, or maybe it's some breaking change and 2.3.x works with the latest version of the IB image? that part I didn't test.)

Provide any steps possible used to reproduce the error (ideally in an isolated fashion).

1. As a Mac user ensure you have gpg 2.2.27

2. Use these steps to generate a GPG keypair, configure .sops.yaml, make a secrets.enc.yaml that flux will be ready to decrypt. https://repo1.dso.mil/platform-one/onboarding/big-bang/engineering-cohort/-/blob/master/lab_guides/05-big-bang-prep-work/C-provision-gpg-and-create-secrets.md

3. Use these commands to install flux https://repo1.dso.mil/platform-one/onboarding/big-bang/engineering-cohort/-/blob/master/lab_guides/06-big-bang-configuration-and-deployment/A-install-flux.md

4. (Tweak the customer template to your git repo IP) https://repo1.dso.mil/platform-one/onboarding/big-bang/engineering-cohort/-/blob/master/lab_guides/06-big-bang-configuration-and-deployment/C-debug-flux-gitrepository-crd.md

What you'll see is that although you have done all the GPG/SOPS/Flux config steps correct
k describe kustomization environment -n=bigbang
Will show an error message like this:

Group 0: FAILED
  CEB7F6F3F629F774608360DE198438480DB3E1E9: FAILED
    - | could not decrypt data key with PGP key:
      | golang.org/x/crypto/openpgp error: Could not load secring:
      | open /tmp/decryptor-environment-022459279/secring.gpg: no
      | such file or directory; GPG binary error: exit status 2

Steps to resolve the error message:

1. Don't recreate the flux sops-gpg secret, there's nothing wrong with it
2. Don't edit the gpg encrypted secret in the git repo, there's nothing wrong with it
3. Downgrade to an earlier version of IronBank Flux and everything works (which makes me think latest IB flux Kustomization Controller is broken).

Latest IB Flux was originally installed via this method:

cd ~/Desktop
git clone https://repo1.dso.mil/platform-one/big-bang/bigbang.git
cd bigbang/
./scripts/install_flux.sh -u $REGISTRY1_USER -p $REGISTRY1_TOKEN

kubectl get deploy -n=flux-system -o wide
# IMAGES                                                           
# registry1.dso.mil/ironbank/fluxcd/helm-controller:v0.11.0        
# registry1.dso.mil/ironbank/fluxcd/kustomize-controller:v0.13.0   
# registry1.dso.mil/ironbank/fluxcd/notification-controller:v0.15.0
# registry1.dso.mil/ironbank/fluxcd/source-controller:v0.14.0      

IB Flux was downgraded via this method, and everything immediately started working again:

cd ~/Desktop/bigbang
git reset --hard d35aae0a # Some older commit I looked up in the GUI
./scripts/install_flux.sh -u $REGISTRY1_USER -p $REGISTRY1_TOKEN

kubectl get deploy -n=flux-system -o wide
# IMAGES                                                           
# registry1.dso.mil/ironbank/fluxcd/helm-controller:v0.9.0         
# registry1.dso.mil/ironbank/fluxcd/kustomize-controller:v0.9.3    
# registry1.dso.mil/ironbank/fluxcd/notification-controller:v0.12.0
# registry1.dso.mil/ironbank/fluxcd/source-controller:v0.9.1       

k describe kustomization environment -n=bigbang 
(Error message disappears, Everything works)

BigBang Version

Irrelevant, not an issue with BigBang, issue with BB Dependency of Flux Controller / Issue with IB Flux Install script that exists in the repo.

Maybe this needs an upstream flux bug report, can someone do that for me? and link it to this, if no one gets to it b4 I'll follow up on Friday.

AC

Either fix bug or document and point to bug in upstream.