diff --git a/docs/README.md b/docs/README.md index 5d827dfea8d38c973c8980965850e89f6bdd1a1e..d5ff4ff3014c2d083ea2a7ccf95765b8a531444d 100644 --- a/docs/README.md +++ b/docs/README.md @@ -6,11 +6,11 @@ * The BigBang Helm Chart deploys gitrepository and helmrelease Custom Resources to a Kubernetes Cluster that's running the Flux GitOps Operator, these can be seen using `kubectl get gitrepository,helmrelease -n=bigbang`. Flux then installs the helm charts defined by the Custom Resources into the cluster. * The BigBang Helm Chart has a values.yaml file that does 2 main things: 1. Defines which DevSecOps Platform packages/helm charts will be deployed - 2. Defines what input parameters will be passed through to the chosen helm charts. + 1. Defines what input parameters will be passed through to the chosen helm charts. * You can see what applications are part of the platform by checking the following resources: - * [../Packages.md](../Packages.md) lists the packages and organizes them in categories. + * [../Packages.md](../Packages.md) lists the packages and organizes them in categories. * [Release Notes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/releases) lists the packages and their versions. - * For a code based source of truth, you can check [BigBang's default values.yaml](../chart/values.yaml), and `[CTRL] + [F]` "repo:", to quickly iterate through the list of applications supported by the BigBang team. + * For a code based source of truth, you can check [BigBang's default values.yaml](../chart/values.yaml), and `[CTRL] + [F]` "repo:", to quickly iterate through the list of applications supported by the BigBang team. ## How do I deploy BigBang? @@ -23,7 +23,7 @@ The following is a general overview of the process, the [deployment guides](guid * Install Flux GitOps Operator on the Cluster. * Configure Flux, the Cluster, and the Git Repo for GitOps Deployments that support deploying encrypted values. * Commit to the Git Repo BigBang's values.yaml and encrypted secrets that have been configured to match the desired state of the cluster (including HTTPS Certs and DNS names). -2. `kubectl apply --filename bigbang.yaml` +1. `kubectl apply --filename bigbang.yaml` * [bigbang.yaml](https://repo1.dso.mil/platform-one/big-bang/customers/template/-/blob/main/dev/bigbang.yaml) will trigger a chain reaction of GitOps Custom Resources' that will deploy other GitOps CR's that will eventually deploy an instance of a DevSecOps Platform that's declaratively defined in your Git Repo. * To be specific, the chain reaction pattern we consider best practice is to have: * bigbang.yaml deploys a gitrepository and kustomization Custom Resource diff --git a/docs/configuration.md b/docs/configuration.md index bdd0df1428a9abb6ef57e1225f12a712ca1774b9..cbbbc289bcefa8ceabb930f5263977dba589e385 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -1,21 +1,6 @@ # Big Bang Configuration -Table of Contents - -- [Big Bang Configuration](#big-bang-configuration) - - [Overview](#overview) - - [Pre-configuration](#pre-configuration) - - [Minimum Viable Configuration](#minimum-viable-configuration) - - [Big Bang Globals](#big-bang-globals) - - [`hostname`](#hostname) - - [`registryCredentials`](#registrycredentials) - - [`flux`](#flux) - - [Package](#package) - - [Flux Resources](#flux-resources) - - [Big Bang Version](#big-bang-version) - - [Environment Location](#environment-location) - - [Registry Pull Credentials](#registry-pull-credentials) - - [Package settings](#package-settings) +[[_TOC_]] ## Overview @@ -121,7 +106,7 @@ Big Bang deploys four flux resources that can be customized: In addition, each package contains its own GitRepository and HelmRelease resource that can be customized. Look in the [Helm chart templates](../chart/templates) for the these resources. -Settings for eny of these resources can be overridden by [patching](https://kubectl.docs.kubernetes.io/references/kustomize/patches/) the resource in your environment's kustomization files. Use Flux's documentation for [GitRepository](https://toolkit.fluxcd.io/components/source/gitrepositories/), [HelmRelease](https://toolkit.fluxcd.io/components/helm/helmreleases/), and [Kustomization](https://toolkit.fluxcd.io/components/kustomize/kustomization/) to find settings for these resources. The following are examples of commonly reqeusted custom patches covered in the [bigbang template repo]https://repo1.dso.mil/platform-one/big-bang/customers/template/-/tree/main#flux-components): +Settings for eny of these resources can be overridden by [patching](https://kubectl.docs.kubernetes.io/references/kustomize/patches/) the resource in your environment's kustomization files. Use Flux's documentation for [GitRepository](https://toolkit.fluxcd.io/components/source/gitrepositories/), [HelmRelease](https://toolkit.fluxcd.io/components/helm/helmreleases/), and [Kustomization](https://toolkit.fluxcd.io/components/kustomize/kustomization/) to find settings for these resources. The following are examples of commonly requested custom patches covered in the [bigbang template repo]<https://repo1.dso.mil/platform-one/big-bang/customers/template/-/tree/main#flux-components>): - Updating flux-system component resource usage - [Example `kustomization.yaml`](https://repo1.dso.mil/platform-one/big-bang/customers/template/-/tree/main#adjust-resource-allocation-for-a-flux-system-component) @@ -133,6 +118,7 @@ Settings for eny of these resources can be overridden by [patching](https://kube - Changing the image name / version - [Example `kustomization.yaml`](https://repo1.dso.mil/platform-one/big-bang/customers/template/-/tree/main#updating-a-flux-system-component-image-tag) - This patch could be used to update the tag of the flux-system component image to be deployed. + > NOTE: Multiple patches could be applied within a single kustomization.yaml ## Big Bang Version diff --git a/docs/deployment.md b/docs/deployment.md index 06ffd229f5369fd88824a4b71a66f66f5af122d5..2616420f0116ab50d7f9d377877b15e5885e0832 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -1,12 +1,6 @@ # Big Bang Deployment -Table of Contents - -- [Big Bang Deployment](#big-bang-deployment) - - [GitOps](#gitops) - - [Installation](#installation) - - [Upgrade](#upgrade) - - [Monitor](#monitor) +[[_TOC_]] ## GitOps @@ -182,7 +176,8 @@ The following commands will help you monitor the progress of the Big Bang deploy gatekeeper-system deployment.apps/gatekeeper-audit 1/1 1 1 2m8s istio-operator deployment.apps/istio-operator 0/1 1 0 8s - NAMESPACE NAME READY STATUS RESTARTS AGEkube-system pod/local-path-provisioner-6d59f47c7-s6rln 1/1 Running 0 4m36s + NAMESPACE NAME READY STATUS RESTARTS AGE + kube-system pod/local-path-provisioner-6d59f47c7-s6rln 1/1 Running 0 4m36s kube-system pod/coredns-7944c66d8d-flk4p 1/1 Running 0 4m36s flux-system pod/helm-controller-578cdbcd8b-tjzs7 1/1 Running 0 4m6s flux-system pod/notification-controller-7c59d85f77-92ckv 1/1 Running 0 4m6s diff --git a/docs/encryption.md b/docs/encryption.md index 4f9e6fb75d178912800d2934c38c649d504c93b3..2c35b700d2b2b57ed438641a40b8281acf1a4f55 100644 --- a/docs/encryption.md +++ b/docs/encryption.md @@ -1,20 +1,6 @@ # Big Bang Encryption -Table of Contents - -- [Big Bang Encryption](#big-bang-encryption) - - [SOPS](#sops) - - [Create Encryption Keys](#create-encryption-keys) - - [Configure SOPS](#configure-sops) - - [Deploy Private Key](#deploy-private-key) - - [GPG](#gpg) - - [AWS KMS](#aws-kms) - - [GCP KMS](#gcp-kms) - - [Azure KeyVault](#azure-keyvault) - - [HashiCorp Vault](#hashicorp-vault) - - [Configure Big Bang](#configure-big-bang) - - [GPG](#gpg-1) - - [AWS KMS](#aws-kms-1) +[[_TOC_]] Big Bang follows a [GitOps](https://www.weave.works/technologies/gitops/) approach to managing the Big Bang Kubernetes cluster configuration. Using GitOps, we must securely store secrets in Git using encryption. The private key, which is stored in key storage, is used by the continuous deployment tool to decrypt and deploy the secrets for use in the cluster. @@ -38,7 +24,6 @@ To setup Big Bang with SOPS, a key pair must be created. The private key is use > *GPG is not recommended for production use because the private key can be misplaced or compromised too easily - ## Configure SOPS SOPS uses `.sops.yaml` as a configuration file for which keys to use for newly created files. Once a file is created, the key fingerprints are stored in the file and must be re-keyed to use any changes to `.sops.yaml`. @@ -56,7 +41,7 @@ SOPS uses `.sops.yaml` as a configuration file for which keys to use for newly c 3. Save `.sops.yaml` in the root of folder of your configuration 4. If you have existing secrets, use the following to re-key them with the configuration in `.sops.yaml` - ```bash + ```shell # You must have the old private key to rekey the file sops updatekeys <encrypted file> ``` @@ -69,25 +54,28 @@ SOPS uses `.sops.yaml` as a configuration file for which keys to use for newly c 1. Deploy your SOPS private key to a secret named `sops-gpg` in the cluster - ```bash + ```shell gpg --export-secret-keys --armor <new key fingerprint> | kubectl create secret generic sops-gpg -n bigbang --from-file=yourkey.asc=/dev/stdin ``` ### AWS KMS 1. Configure your KMS key(s) in your `.sops.yaml` by adding the target key's ARN to the `kms` field within each creation rule. + ```yaml creation_rules: - encrypted_regex: '^(data|stringData)$' path_regex: ./dev/.* kms: '<kms_key_arn>' ``` -2. Ensure your cluster (specifically the `flux-system/flux-controller`) has access to the specified key. + +1. Ensure your cluster (specifically the `flux-system/flux-controller`) has access to the specified key. 1. For AWS deployments, this can be managed via IAM roles as described in the [SOPS documentation](https://github.com/mozilla/sops#28assuming-roles-and-using-kms-in-various-aws-accounts). - 2. For non-AWS deployments + 1. For non-AWS deployments 1. Create an AWS user with appropriate permissions as described in the [SOPS documentation](https://github.com/mozilla/sops#28assuming-roles-and-using-kms-in-various-aws-accounts). - 2. Create a secret named `sops-aws-creds` in the cluster using the access creds from the target user: - ```bash + 1. Create a secret named `sops-aws-creds` in the cluster using the access creds from the target user: + + ```shell k create secret generic -n flux-system sops-aws-creds --from-literal=access_key_id=<key_id> --from-literal=access_key_secret=<key> ``` @@ -105,10 +93,12 @@ TBD - [This article](https://blog.doit-intl.com/injecting-secrets-from-aws-gcp-o ## Configure Big Bang -Big Bang needs to know how to retrieve the private key so it can deploy the encrypted secrets from Git. Decryption configuration is placed in the top-level manifest (e.g. `dev.yaml`, `prod.yaml`) from the [Big Bang template](https://repo1.dso.mil/platform-one/big-bang/customers/template). +Big Bang needs to know how to retrieve the private key so it can deploy the encrypted secrets from Git. Decryption configuration is placed in the top-level manifest (e.g. `dev.yaml`, `prod.yaml`) from the [Big Bang template](https://repo1.dso.mil/platform-one/big-bang/customers/template). ### GPG + By default, the `Kustomization` resource uses a Secret named `sops-gpg` for the private key as shown here: + ```yaml apiVersion: kustomize.toolkit.fluxcd.io/v1beta1 kind: Kustomization @@ -122,7 +112,9 @@ spec: ``` ### AWS KMS + Configure the `Kustomization` resource to use sops for decryption: + ```yaml apiVersion: kustomize.toolkit.fluxcd.io/v1beta1 kind: Kustomization @@ -132,12 +124,14 @@ spec: decryption: provider: sops ``` + > Note, we are not providing the `secretRef` field, which is specific to GPG -If Big Bang is deployed within AWS, KMS key access can be handled via IAM roles and permissions on the cluster resources themselves. -However, if the deployment is in a different environment from the KMS keys, AWS credentials may need to be provided via a secret as follows. +If Big Bang is deployed within AWS, KMS key access can be handled via IAM roles and permissions on the cluster resources themselves. +However, if the deployment is in a different environment from the KMS keys, AWS credentials may need to be provided via a secret as follows. Configure the flux-system `kustomize-controller` component with AWS credential environment variables using `kustomize`. Specific instructions for doing this may vary by deployment and environment but [an example](https://repo1.dso.mil/platform-one/big-bang/customers/template/-/tree/master#flux-components) is covered in the bigbang template repo. Broadly speaking, adding environment variables to the `kustomize-controller` component can be accomplished by adding a patch to the `flux/kustomization.yaml` for the target deployment or environment. An example of such a `kustomization.yaml` is shown below: + ```yaml bases: - ../../base/flux @@ -166,6 +160,7 @@ patchesStrategicMerge: name: sops-aws-creds key: access_key_secret ``` + > Values should come from the `sops-aws-creds` secret created in [AWS KMS](#aws-kms) above TBD - Instructions on how to update for GCP, Vault diff --git a/docs/overview.md b/docs/overview.md index 8a650f1e6892409987f68704fdcbe36985698eb6..524ab4074da093ff7b9ffb1104ba90cb9414ef84 100644 --- a/docs/overview.md +++ b/docs/overview.md @@ -1,21 +1,6 @@ # Big Bang Overview -Table of Contents - -- [Big Bang Overview](#big-bang-overview) - - [Key Concepts](#key-concepts) - - [Big Bang](#big-bang) - - [Big Bang Components](#big-bang-components) - - [Flux v2](#flux-v2) - - [GitOps](#gitops) - - [SOPS](#sops) - - [Kustomize](#kustomize) - - [Helm](#helm) - - [Packages](#packages) - - [Architecture](#architecture) - - [Configuration](#configuration) - - [Deployment](#deployment) - - [Diagram](#diagram) +[[_TOC_]] --- diff --git a/docs/production.md b/docs/production.md index b3e1769447f5e30bfc5ad1420b3c529596bd7f24..035b6435351a874e7543ee3e40da61a3a49aaff7 100644 --- a/docs/production.md +++ b/docs/production.md @@ -1,14 +1,12 @@ # Big Bang Production -Table of Contents - -- [Big Bang Production](#big-bang-production) - - [Production Deployment](#production-deployment) +[[_TOC_]] ## Production Deployment The gatekeeper `values` section should resemble below when deploying to production. -``` + +```yaml # OPA Gatekeeper # gatekeeper: @@ -41,7 +39,8 @@ To validate it was deployed correctly on your cluster run the following command: You should only see `kube-system` under `excludedNamespaces` section. Output: -``` + +```yaml name: allowed-docker-registries resourceVersion: "10390" uid: b51b3887-3cf8-4495-b37e-fb8ef31755db @@ -61,5 +60,3 @@ spec: - registry1.dso.mil - registry.dso.mil ``` - -