[P1BIGROCKS-2798] Big Bang Command Line Interface (CLI) to simplify development, deployment, auditing, and troubleshooting

P1BIGROCKS-2798

MVP1: BBOOI-1158

Problem:

• Big Bang is very complex and can be difficult for new users or users not familiar with Kubernetes to develop, deploy, and use
• Following Big Bang documented procedures can be time consuming and are subject to human error. Documentation also becomes outdated quickly.
• Scripts have been created to help with Big Bang's complexity. However, they are dispersed throughout Big Bang and can be difficult to find/use.
• Synergy between Big Bang's scripts are minimal and sometimes offer different solutions to similar problems
• Current scripts are written for Linux bash and do not always port to other shells (e.g. Mac bash, zsh)
• Troubleshooting and/or auditing Big Bang requires deep knowledge of other command line tools like kubectl, istioctl, flux, and jq and the resources deployed by Big Bang or the packages. Many consumers of Big Bang have only a cursory knowledge of these tools and need a simpler way to check for common Big Bang issues.

Proposed Solution:

Consolidate Big Bang procedures for development, deployment, auditing, and troubleshooting into a command-line interface (CLI)

• Architect CLI in the same pattern as kubectl or istioctl to facilitate adoption by users familiar with those tools
• Over time, replace (or replace sections) of scripts with CLI calls
• Create a pipeline to validate the procedures implemented in CLI, which identifies when procedures become outdated
• Use go language to maximize compatibility over OSs/shell
• Integrate extensive options, interaction, and help to simplify user and developer experience
• Consolidating these into a single tool facilitates code reuse and synergy

Possible Commands:

• bbctl init pkg: Create a flux based helm template for a new package like BB's chart/templates//* that can then be customized
• bbctl init dev: Setup default development framework (e.g. scripts) and values for deploying a local or AWS cluster with Big Bang
• bbctl init prod: Setup customer framework that can be customized
• bbctl init airgap: Create registry tarball, git repository tarball, and any other items (Zarf .yaml?) for use in offline deployment
• bbctl config <pkg>: Interactively configure common settings for a package (or Big Bang) and produce the YAML (e.g. database, sso, hostname)
• bbctl install flux: Installs and validate flux deployment
• bbctl install creds: Creates, encrypts, and installs registry or git credentials for use in Big Bang
• bbctl install certs: Creates, encrypts, and installs TLS certs in Big Bang
• bbctl install ca: Creates and installs certificate authorities in Big Bang
• bbctl get violations <pkg>: Display a report of all OPA Gatekeeper violations or denies in the cluster or a specific package
• bbctl get ingress <pkg>: List exposed ingress points, including port, URL, and protocol for the cluster or a specific package
• bbctl get creds <pkg>: List login credentials for a package or all packages (stored in a secret)
• bbctl get status <pkg>: Check status of packages deployed by Big Bang for completion and summarizes results. Includes GitRepository, HelmRelease, operator, custom resources, deployments, persistent volumes, stateful sets, daemon sets, pods, services, etc.
• bbctl get version: Reach out to a cluster and get the BB version and all package versions running
• bbctl get images: Get all images in the cluster's docker registry
• bbctl test lint: Check for common mistakes in Big Bang's configuration files
• bbctl test ingress <pkg>: Check access to all endpoints or specific package endpoints
• bbctl remove <pkg>: Gracefully remove a package (or all of BigBang) from a cluster. Takes into account order of removing custom resource before operator, suspending flux resources, etc. so that you aren't left with dangling resources.
• bbctl sync <pkg>: Kickstart flux to immediately sync GitRepository, HelmRelease, etc for a package.

Each one of these could be it's own issue to breakdown.

MVP1 Epic Link

Version 2

• Installation of Big Bang
• Upgrades for Big Bang from version x to x.

Open Questions

• Would upgrading packages (e.g. special procedures) or progressive delivery (e.g. canary, blue/green) be included in this?
  • Something to think about for version 2. MVP should be about getting statuses.
• Do we have the skillset in Big Bang to develop and maintain a Go application?