General MR

Summary

This changes the default behavior of the K3D script to use a MetalLB load balancer. This has apparently been the "most correct" behavior the entire time; this is the behavior of the Big Bang repo1 pipelines, and most closely mirrors the setup used in an EKS deployment. However the k3d-dev script has used a single IP based, single load balancer deployment for the longest time. This causes confusion and issues. This MR resolves this by

forcing the default behavior to use MetalLB.
deprecating the -m option (it's still there for compatibility but does nothing)
adding a -M option for those rare cases where you explicitly want to disable MetalLB

Relevant logs/screenshots

Creating with no flags:

$ bash k3d-dev.sh
# ... snip ...

================================================================================
====================== DEPLOYMENT FINISHED =====================================
================================================================================

SAVE THE FOLLOWING INSTRUCTIONS INTO A TEMPORARY TEXT DOCUMENT SO THAT YOU DON'T LOSE THEM
NOTE: The EC2 instance will automatically terminate 8 hours from the time of creation unless you delete the root cron job

ssh to instance:
  ssh -i /Users/andrewkesterson/.ssh/Andrew.Kesterson-dev-default.pem -o IdentitiesOnly=yes ubuntu@3.32.201.167

To use kubectl from your local workstation you must set the KUBECONFIG environment variable:
  export KUBECONFIG=~/.kube/Andrew.Kesterson-dev-default-config

OPTION 1: ACCESS APPLICATIONS WITH WEB BROWSER ONLY
To access apps from browser only start ssh with application-level port forwarding: 
  ssh -i /Users/andrewkesterson/.ssh/Andrew.Kesterson-dev-default.pem ubuntu@3.32.201.167 -D 127.0.0.1:12345
Do not edit /etc/hosts on your local workstation.
Edit /etc/hosts on the EC2 instance. Sample /etc/host entries have already been added there.
Manually add more hostnames as needed.
The IPs to use come from the istio-system services of type LOADBALANCER EXTERNAL-IP that are created when Istio is deployed.
You must use Firefox browser with with manual SOCKs v5 proxy configuration to localhost with port 12345.
Also ensure 'Proxy DNS when using SOCKS v5' is checked.
Or, with other browsers like Chrome you could use a browser plugin like foxyproxy to do the same thing as Firefox.

OPTION 2: ACCESS APPLICATIONS WITH WEB BROWSER AND COMMAND LINE
To access apps from browser and from the workstation command line start sshuttle in a separate terminal window.
  sshuttle --dns -vr ubuntu@3.32.201.167 172.20.1.0/24 --ssh-cmd 'ssh -i /Users/andrewkesterson/.ssh/Andrew.Kesterson-dev-default.pem'
Edit your workstation /etc/hosts to add the LOADBALANCER EXTERNAL-IPs from the istio-system services with application hostnames.
Here is an example. You might have to change this depending on the number of gateways you configure for k8s cluster.
  # METALLB ISTIO INGRESS IPs
  172.20.1.240 keycloak.dev.bigbang.mil vault.dev.bigbang.mil
  172.20.1.241 sonarqube.dev.bigbang.mil prometheus.dev.bigbang.mil nexus.dev.bigbang.mil gitlab.dev.bigbang.mil

Creating with -M for no MetalLB:

================================================================================                                                                   
====================== DEPLOYMENT FINISHED =====================================                                                                   
================================================================================                                                                   

SAVE THE FOLLOWING INSTRUCTIONS INTO A TEMPORARY TEXT DOCUMENT SO THAT YOU DON'T LOSE THEM
NOTE: The EC2 instance will automatically terminate 8 hours from the time of creation unless you delete the root cron job

ssh to instance:                                                         
  ssh -i /Users/andrewkesterson/.ssh/Andrew.Kesterson-dev-default.pem -o IdentitiesOnly=yes ubuntu@160.1.16.4

To use kubectl from your local workstation you must set the KUBECONFIG environment variable:
  export KUBECONFIG=~/.kube/Andrew.Kesterson-dev-default-config                                                                                    

To access apps from a browser edit your /etc/hosts to add the public IP of your EC2 instance with application hostnames.
Example:                                                                 
  160.1.16.4 gitlab.dev.bigbang.mil prometheus.dev.bigbang.mil kibana.dev.bigbang.mil

Linked Issue

N/A

Upgrade Notices

The default behavior of the k3d-dev script has been changed to better match our CI/CD pipelines and our production EKS setups. The default behavior is now to create a MetalLB load balancer. The -m flag is supported but deprecated and is now the default behavior. If you REQUIRE the old behavior, you can disable metallb with -M.