Update quickstart documentation, make k3d-dev reprint instructions
General MR
Summary
With the recent move to standardize k3d-dev on MetalLB deployments, the quickstart documentation is no longer correct. It was written with the assumption that MetalLB was not used. This MR corrects the documentation to match the default use case.
The quickstart script also issues instructions to the user that are no longer correct. It prints its own instructional message that may not match the arguments given to k3d-dev or k3d-dev's current internal behavior. This MR corrects that by reusing the instructional message from k3d-dev directly, instead of printing any instructions of its own.
This adds a --print-instructions option to k3d-dev which prints the instructional message for the instance referenced on the command line, and performs no other actions. It assumes the instance referenced on the command line already exists, and fails if that assumption does not
appear to be true.
Risks
This feature comes with a small-ish refactor.
To support the printing of accurate instructions independent of the rest of the instance creation/configuration process, some logic had to be moved into check_for_existing_instances function, and this new function had to be called at the top of main after argument were processed and before any actions were taken. This required a refactor of how the question "Are we provisioning a cloud instance or using an instance provided by the user?" is answered. This was previously overloaded onto the $PublicIP variable, but this was no longer feasible with the logic (and population of related values) moved into the check_for_existing_instances function at the top of main. So a new boolean variable has been created, $PROVISION_CLOUD_INSTANCE, and the $PublicIP variable no longer has a boolean purpose overloaded onto it.
Alternative Implementations
This implementation relies on the user providing the exact same command line arguments when calling --print-instructions as when the instance was created. This is convenient for me as the script author, and it works for the initial use case (printing just the instructions from the quickstart), but it seems failure prone for the user. How likely are they to remember how they constructed the instance if they can't remember how to connect to it?
I'm open to either alternate implementation options below, but this MR represents the option that I wrote first, and it's suitable for a MVP, so that's what I'm submitting.
Option 1
The quickest and dirtiest way for the quickstart.sh to reuse the k3d-dev.sh instructional message is to capture the output, and grep out everything that proceeds after the DEPLOYMENT FINISHED line. This would be rudimentary but effective. However it is more fragile than either the implementation here, or Option 2, below. But it would be the smallest possible changeset to accomplish the goal.
Option 2
The script does tell the user to save these instructions to a file for later referencing, but ... the script could also just save the instructions and print them out later. But if we save the instructions to a file and print them out later, we have to figure out where to store the instructions for future reference. That storage location would need to be:
- Platform independent (at the very least it should work on Linux, macOS and Windows running git bash/msys2/WSL)
- Not located on a cloud provider (
k3d-devallows you to manage instances that are not cloud provisioned, so this would need to be on the local user's workstation) - Easy to manage from bash
The simplest option is to just dump text files into a ~/.k3d-dev/ directory or something. There's limited danger in assuming the presence of $HOME in a Windows msys2/git bash environment, but this is probably safe. (msys2/git bash will provide a value for $HOME but it may not be what the user expects - but in this case, it probably doesn't matter. Our contract with the user would be "we will put them in ${HOME}/.k3d-dev/", whatever $HOME has in it.)
The risk with this approach is that, if something happens and the instructions aren't cleaned up when an instance is destroyed, the user might be able to print out (or manually reference) incorrect instructions. But that may be a lesser evil than forcing the user to remember their command line arguments. It would also require a far less significant refactor of the code.
There would also be a slight complication in figuring out how to key the entries in the directory. Cloud instances could be named for their instance IDs and project tags. Non-cloud instances could probably be named after their Public IP and their project tag. I'm sure we would find corner cases where this would fall over, but it's probably sufficient.
Relevant logs/screenshots
The quickstart now duplicates the k3d-dev instructions for connectivity instead of printing its own:
$ quickstart.sh
# ... snip ...
==================================================================================
INSTALLATION COMPLETE
SAVE THE FOLLOWING INSTRUCTIONS INTO A TEMPORARY TEXT DOCUMENT SO THAT YOU DON'T LOSE THEM
ssh to instance:
ssh -i ~/.ssh/Andrew.Kesterson-dev-quickstart.pem -o IdentitiesOnly=yes ubuntu@15.200.110.35
To use kubectl from your local workstation you must set the KUBECONFIG environment variable:
export KUBECONFIG=~/.kube/15.200.110.35-dev-quickstart-config
OPTION 1: ACCESS APPLICATIONS WITH WEB BROWSER ONLY
To access apps from browser only start ssh with application-level port forwarding:
ssh -i ~/.ssh/Andrew.Kesterson-dev-quickstart.pem ubuntu@15.200.110.35 -D 127.0.0.1:12345
Do not edit /etc/hosts on your local workstation.
Edit /etc/hosts on the EC2 instance. Sample /etc/host entries have already been added there.
Manually add more hostnames as needed.
The IPs to use come from the istio-system services of type LOADBALANCER EXTERNAL-IP that are created when Istio is
deployed.
You must use Firefox browser with with manual SOCKs v5 proxy configuration to localhost with port 12345.
Also ensure 'Proxy DNS when using SOCKS v5' is checked.
Or, with other browsers like Chrome you could use a browser plugin like foxyproxy to do the same thing as Firefox.
OPTION 2: ACCESS APPLICATIONS WITH WEB BROWSER AND COMMAND LINE
To access apps from browser and from the workstation command line start sshuttle in a separate terminal window.
sshuttle --dns -vr ubuntu@15.200.110.35 172.20.1.0/24 --ssh-cmd 'ssh -i ~/.ssh/Andrew.Kesterson-dev-quickstart.p
em'
Edit your workstation /etc/hosts to add the LOADBALANCER EXTERNAL-IPs from the istio-system services with applicat
ion hostnames.
Here is an example. You might have to change this depending on the number of gateways you configure for k8s cluste
r.
# METALLB ISTIO INGRESS IPs
172.20.1.240 keycloak.dev.bigbang.mil vault.dev.bigbang.mil
172.20.1.241 sonarqube.dev.bigbang.mil prometheus.dev.bigbang.mil nexus.dev.bigbang.mil gitlab.dev.bigbang.mil
==================================================================================Here is the bare k3d-dev.sh provisioning an instance and printing instructions:
$ k3d-dev.sh -m -a
# ... snip ...
SAVE THE FOLLOWING INSTRUCTIONS INTO A TEMPORARY TEXT DOCUMENT SO THAT YOU DON'T LOSE THEM
NOTE: The EC2 instance will automatically terminate 8 hours from the time of creation unless you delete the root cron job
ssh to instance:
ssh -i /Users/andrewkesterson/.ssh/Andrew.Kesterson-dev-default.pem -o IdentitiesOnly=yes ubuntu@3.30.57.238 15.205.246.177
To use kubectl from your local workstation you must set the KUBECONFIG environment variable:
export KUBECONFIG=~/.kube/Andrew.Kesterson-dev-default-config
To access apps from a browser edit your /etc/hosts to add the public IP of your EC2 instance with application hostnames.
Example:
3.30.57.238 15.205.246.177 gitlab.dev.bigbang.mil prometheus.dev.bigbang.mil kibana.dev.bigbang.mil
A secondary IP is available for use if you wish to have a passthrough ingress for Istio along with a public Ingress Gateway, this maybe useful for Keycloak x509 mTLS authentication.
56.136.53.119 keycloak.dev.bigbang.milHere are the instructions being regenerated for that same instance later:
$ k3d-dev.sh -m -a --print-instructions
SAVE THE FOLLOWING INSTRUCTIONS INTO A TEMPORARY TEXT DOCUMENT SO THAT YOU DON'T LOSE THEM
NOTE: The EC2 instance will automatically terminate 8 hours from the time of creation unless you delete the root cron job
ssh to instance:
ssh -i /Users/andrewkesterson/.ssh/Andrew.Kesterson-dev-default.pem -o IdentitiesOnly=yes ubuntu@3.30.57.238 15.205.246.177
To use kubectl from your local workstation you must set the KUBECONFIG environment variable:
export KUBECONFIG=~/.kube/Andrew.Kesterson-dev-default-config
To access apps from a browser edit your /etc/hosts to add the public IP of your EC2 instance with application hostnames.
Example:
3.30.57.238 15.205.246.177 gitlab.dev.bigbang.mil prometheus.dev.bigbang.mil kibana.dev.bigbang.mil
A secondary IP is available for use if you wish to have a passthrough ingress for Istio along with a public Ingress Gateway, this maybe useful for Keycloak x509 mTLS authentication.
56.136.53.119 keycloak.dev.bigbang.mil... Attempting to reprint the instructions for an instance doesn't exist:
$ k3d-dev.sh -m -a --print-instructions
There does not appear to be a valid existing instance matching the arguments provided.
Cannot generate instructions.This shows that, when using --project-tag|-t to run multiple instances, the --print-instructions method prints correct instructions for each instance (assuming the correct flags, like -m or -a etc, are provided on the command line):
$ k3d-dev.sh -r
i-05997db66dfb4b570 running 15.200.29.212 sg-06428d01f8554792b private_test_instance
i-041bb256c27f9b3a6 running 52.61.28.203 sg-0f8efce363f98c1ec defaultShowing the instructions for the private_test_instance
$ k3d-dev.sh --use-private-ip -t private_test_instance --print-instructions
SAVE THE FOLLOWING INSTRUCTIONS INTO A TEMPORARY TEXT DOCUMENT SO THAT YOU DON'T LOSE THEM
NOTE: The EC2 instance will automatically terminate 8 hours from the time of creation unless you delete the root cron job
ssh to instance:
ssh -i /Users/andrewkesterson/.ssh/Andrew.Kesterson-dev-private_test_instance.pem -o IdentitiesOnly=yes ubuntu@15.200.29.212
To use kubectl from your local workstation you must set the KUBECONFIG environment variable:
export KUBECONFIG=~/.kube/Andrew.Kesterson-dev-private_test_instance-config
The cluster connection will not work until you start sshuttle as described below.
Start sshuttle in a separate terminal window:
sshuttle --dns -vr ubuntu@15.200.29.212 172.31.0.0/16 --ssh-cmd 'ssh -i /Users/andrewkesterson/.ssh/Andrew.Kesterson-dev-private_test_instance.pem -D 127.0.0.1:12345'
Do not edit /etc/hosts on your local workstation.
# ... snip ...Showing the instructions for the default instance
$ k3d-dev.sh -m -a --print-instructions
SAVE THE FOLLOWING INSTRUCTIONS INTO A TEMPORARY TEXT DOCUMENT SO THAT YOU DON'T LOSE THEM
NOTE: The EC2 instance will automatically terminate 8 hours from the time of creation unless you delete the root cron job
ssh to instance:
ssh -i /Users/andrewkesterson/.ssh/Andrew.Kesterson-dev-default.pem -o IdentitiesOnly=yes ubuntu@52.61.28.203
To use kubectl from your local workstation you must set the KUBECONFIG environment variable:
export KUBECONFIG=~/.kube/Andrew.Kesterson-dev-default-config
To access apps from a browser edit your /etc/hosts to add the public IP of your EC2 instance with application hostnames.
Example:
52.61.28.203 gitlab.dev.bigbang.mil prometheus.dev.bigbang.mil kibana.dev.bigbang.milLinked Issue
NA
Upgrade Notices
Quickstart users should pay attention to the instructions printed at the end of the quickstart script, as they have changed.
Merge request reports
Activity
changed milestone to %2.47.0
added statusreview label
assigned to @akesterson
requested review from @michaelmartin, @chris.oconnell, @andrewshoell, and @troymobley
changed milestone to %2.48.0
added 5 commits
-
e3af12f0...89f0bd93 - 4 commits from branch
master - 135a5350 - Update quickstart documentation
-
e3af12f0...89f0bd93 - 4 commits from branch