UNCLASSIFIED - NO CUI

Skip to content
Snippets Groups Projects
Commit 58ac76d9 authored by Jason Krause's avatar Jason Krause :8ball: Committed by Micah Nagel
Browse files

Top-Level Docs updates

parent d5463c2e
No related branches found
No related tags found
2 merge requests!1658Draft: Merge branch 'tempo_tracing_updates' into 'master',!1445Top-Level Docs updates
......@@ -6,11 +6,11 @@
* The BigBang Helm Chart deploys gitrepository and helmrelease Custom Resources to a Kubernetes Cluster that's running the Flux GitOps Operator, these can be seen using `kubectl get gitrepository,helmrelease -n=bigbang`. Flux then installs the helm charts defined by the Custom Resources into the cluster.
* The BigBang Helm Chart has a values.yaml file that does 2 main things:
1. Defines which DevSecOps Platform packages/helm charts will be deployed
2. Defines what input parameters will be passed through to the chosen helm charts.
1. Defines what input parameters will be passed through to the chosen helm charts.
* You can see what applications are part of the platform by checking the following resources:
* [../Packages.md](../Packages.md) lists the packages and organizes them in categories.
* [../Packages.md](../Packages.md) lists the packages and organizes them in categories.
* [Release Notes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/releases) lists the packages and their versions.
* For a code based source of truth, you can check [BigBang's default values.yaml](../chart/values.yaml), and `[CTRL] + [F]` "repo:", to quickly iterate through the list of applications supported by the BigBang team.
* For a code based source of truth, you can check [BigBang's default values.yaml](../chart/values.yaml), and `[CTRL] + [F]` "repo:", to quickly iterate through the list of applications supported by the BigBang team.
## How do I deploy BigBang?
......@@ -23,7 +23,7 @@ The following is a general overview of the process, the [deployment guides](guid
* Install Flux GitOps Operator on the Cluster.
* Configure Flux, the Cluster, and the Git Repo for GitOps Deployments that support deploying encrypted values.
* Commit to the Git Repo BigBang's values.yaml and encrypted secrets that have been configured to match the desired state of the cluster (including HTTPS Certs and DNS names).
2. `kubectl apply --filename bigbang.yaml`
1. `kubectl apply --filename bigbang.yaml`
* [bigbang.yaml](https://repo1.dso.mil/platform-one/big-bang/customers/template/-/blob/main/dev/bigbang.yaml) will trigger a chain reaction of GitOps Custom Resources' that will deploy other GitOps CR's that will eventually deploy an instance of a DevSecOps Platform that's declaratively defined in your Git Repo.
* To be specific, the chain reaction pattern we consider best practice is to have:
* bigbang.yaml deploys a gitrepository and kustomization Custom Resource
......
# Big Bang Configuration
Table of Contents
- [Big Bang Configuration](#big-bang-configuration)
- [Overview](#overview)
- [Pre-configuration](#pre-configuration)
- [Minimum Viable Configuration](#minimum-viable-configuration)
- [Big Bang Globals](#big-bang-globals)
- [`hostname`](#hostname)
- [`registryCredentials`](#registrycredentials)
- [`flux`](#flux)
- [Package](#package)
- [Flux Resources](#flux-resources)
- [Big Bang Version](#big-bang-version)
- [Environment Location](#environment-location)
- [Registry Pull Credentials](#registry-pull-credentials)
- [Package settings](#package-settings)
[[_TOC_]]
## Overview
......@@ -121,7 +106,7 @@ Big Bang deploys four flux resources that can be customized:
In addition, each package contains its own GitRepository and HelmRelease resource that can be customized. Look in the [Helm chart templates](../chart/templates) for the these resources.
Settings for eny of these resources can be overridden by [patching](https://kubectl.docs.kubernetes.io/references/kustomize/patches/) the resource in your environment's kustomization files. Use Flux's documentation for [GitRepository](https://toolkit.fluxcd.io/components/source/gitrepositories/), [HelmRelease](https://toolkit.fluxcd.io/components/helm/helmreleases/), and [Kustomization](https://toolkit.fluxcd.io/components/kustomize/kustomization/) to find settings for these resources. The following are examples of commonly reqeusted custom patches covered in the [bigbang template repo]https://repo1.dso.mil/platform-one/big-bang/customers/template/-/tree/main#flux-components):
Settings for eny of these resources can be overridden by [patching](https://kubectl.docs.kubernetes.io/references/kustomize/patches/) the resource in your environment's kustomization files. Use Flux's documentation for [GitRepository](https://toolkit.fluxcd.io/components/source/gitrepositories/), [HelmRelease](https://toolkit.fluxcd.io/components/helm/helmreleases/), and [Kustomization](https://toolkit.fluxcd.io/components/kustomize/kustomization/) to find settings for these resources. The following are examples of commonly requested custom patches covered in the [bigbang template repo]<https://repo1.dso.mil/platform-one/big-bang/customers/template/-/tree/main#flux-components>):
- Updating flux-system component resource usage
- [Example `kustomization.yaml`](https://repo1.dso.mil/platform-one/big-bang/customers/template/-/tree/main#adjust-resource-allocation-for-a-flux-system-component)
......@@ -133,6 +118,7 @@ Settings for eny of these resources can be overridden by [patching](https://kube
- Changing the image name / version
- [Example `kustomization.yaml`](https://repo1.dso.mil/platform-one/big-bang/customers/template/-/tree/main#updating-a-flux-system-component-image-tag)
- This patch could be used to update the tag of the flux-system component image to be deployed.
> NOTE: Multiple patches could be applied within a single kustomization.yaml
## Big Bang Version
......
# Big Bang Deployment
Table of Contents
- [Big Bang Deployment](#big-bang-deployment)
- [GitOps](#gitops)
- [Installation](#installation)
- [Upgrade](#upgrade)
- [Monitor](#monitor)
[[_TOC_]]
## GitOps
......@@ -182,7 +176,8 @@ The following commands will help you monitor the progress of the Big Bang deploy
gatekeeper-system deployment.apps/gatekeeper-audit 1/1 1 1 2m8s
istio-operator deployment.apps/istio-operator 0/1 1 0 8s
NAMESPACE NAME READY STATUS RESTARTS AGEkube-system pod/local-path-provisioner-6d59f47c7-s6rln 1/1 Running 0 4m36s
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/local-path-provisioner-6d59f47c7-s6rln 1/1 Running 0 4m36s
kube-system pod/coredns-7944c66d8d-flk4p 1/1 Running 0 4m36s
flux-system pod/helm-controller-578cdbcd8b-tjzs7 1/1 Running 0 4m6s
flux-system pod/notification-controller-7c59d85f77-92ckv 1/1 Running 0 4m6s
......
# Big Bang Encryption
Table of Contents
- [Big Bang Encryption](#big-bang-encryption)
- [SOPS](#sops)
- [Create Encryption Keys](#create-encryption-keys)
- [Configure SOPS](#configure-sops)
- [Deploy Private Key](#deploy-private-key)
- [GPG](#gpg)
- [AWS KMS](#aws-kms)
- [GCP KMS](#gcp-kms)
- [Azure KeyVault](#azure-keyvault)
- [HashiCorp Vault](#hashicorp-vault)
- [Configure Big Bang](#configure-big-bang)
- [GPG](#gpg-1)
- [AWS KMS](#aws-kms-1)
[[_TOC_]]
Big Bang follows a [GitOps](https://www.weave.works/technologies/gitops/) approach to managing the Big Bang Kubernetes cluster configuration. Using GitOps, we must securely store secrets in Git using encryption. The private key, which is stored in key storage, is used by the continuous deployment tool to decrypt and deploy the secrets for use in the cluster.
......@@ -38,7 +24,6 @@ To setup Big Bang with SOPS, a key pair must be created. The private key is use
> *GPG is not recommended for production use because the private key can be misplaced or compromised too easily
## Configure SOPS
SOPS uses `.sops.yaml` as a configuration file for which keys to use for newly created files. Once a file is created, the key fingerprints are stored in the file and must be re-keyed to use any changes to `.sops.yaml`.
......@@ -56,7 +41,7 @@ SOPS uses `.sops.yaml` as a configuration file for which keys to use for newly c
3. Save `.sops.yaml` in the root of folder of your configuration
4. If you have existing secrets, use the following to re-key them with the configuration in `.sops.yaml`
```bash
```shell
# You must have the old private key to rekey the file
sops updatekeys <encrypted file>
```
......@@ -69,25 +54,28 @@ SOPS uses `.sops.yaml` as a configuration file for which keys to use for newly c
1. Deploy your SOPS private key to a secret named `sops-gpg` in the cluster
```bash
```shell
gpg --export-secret-keys --armor <new key fingerprint> | kubectl create secret generic sops-gpg -n bigbang --from-file=yourkey.asc=/dev/stdin
```
### AWS KMS
1. Configure your KMS key(s) in your `.sops.yaml` by adding the target key's ARN to the `kms` field within each creation rule.
```yaml
creation_rules:
- encrypted_regex: '^(data|stringData)$'
path_regex: ./dev/.*
kms: '<kms_key_arn>'
```
2. Ensure your cluster (specifically the `flux-system/flux-controller`) has access to the specified key.
1. Ensure your cluster (specifically the `flux-system/flux-controller`) has access to the specified key.
1. For AWS deployments, this can be managed via IAM roles as described in the [SOPS documentation](https://github.com/mozilla/sops#28assuming-roles-and-using-kms-in-various-aws-accounts).
2. For non-AWS deployments
1. For non-AWS deployments
1. Create an AWS user with appropriate permissions as described in the [SOPS documentation](https://github.com/mozilla/sops#28assuming-roles-and-using-kms-in-various-aws-accounts).
2. Create a secret named `sops-aws-creds` in the cluster using the access creds from the target user:
```bash
1. Create a secret named `sops-aws-creds` in the cluster using the access creds from the target user:
```shell
k create secret generic -n flux-system sops-aws-creds --from-literal=access_key_id=<key_id> --from-literal=access_key_secret=<key>
```
......@@ -105,10 +93,12 @@ TBD - [This article](https://blog.doit-intl.com/injecting-secrets-from-aws-gcp-o
## Configure Big Bang
Big Bang needs to know how to retrieve the private key so it can deploy the encrypted secrets from Git. Decryption configuration is placed in the top-level manifest (e.g. `dev.yaml`, `prod.yaml`) from the [Big Bang template](https://repo1.dso.mil/platform-one/big-bang/customers/template).
Big Bang needs to know how to retrieve the private key so it can deploy the encrypted secrets from Git. Decryption configuration is placed in the top-level manifest (e.g. `dev.yaml`, `prod.yaml`) from the [Big Bang template](https://repo1.dso.mil/platform-one/big-bang/customers/template).
### GPG
By default, the `Kustomization` resource uses a Secret named `sops-gpg` for the private key as shown here:
```yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1beta1
kind: Kustomization
......@@ -122,7 +112,9 @@ spec:
```
### AWS KMS
Configure the `Kustomization` resource to use sops for decryption:
```yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1beta1
kind: Kustomization
......@@ -132,12 +124,14 @@ spec:
decryption:
provider: sops
```
> Note, we are not providing the `secretRef` field, which is specific to GPG
If Big Bang is deployed within AWS, KMS key access can be handled via IAM roles and permissions on the cluster resources themselves.
However, if the deployment is in a different environment from the KMS keys, AWS credentials may need to be provided via a secret as follows.
If Big Bang is deployed within AWS, KMS key access can be handled via IAM roles and permissions on the cluster resources themselves.
However, if the deployment is in a different environment from the KMS keys, AWS credentials may need to be provided via a secret as follows.
Configure the flux-system `kustomize-controller` component with AWS credential environment variables using `kustomize`. Specific instructions for doing this may vary by deployment and environment but [an example](https://repo1.dso.mil/platform-one/big-bang/customers/template/-/tree/master#flux-components) is covered in the bigbang template repo. Broadly speaking, adding environment variables to the `kustomize-controller` component can be accomplished by adding a patch to the `flux/kustomization.yaml` for the target deployment or environment. An example of such a `kustomization.yaml` is shown below:
```yaml
bases:
- ../../base/flux
......@@ -166,6 +160,7 @@ patchesStrategicMerge:
name: sops-aws-creds
key: access_key_secret
```
> Values should come from the `sops-aws-creds` secret created in [AWS KMS](#aws-kms) above
TBD - Instructions on how to update for GCP, Vault
# Big Bang Overview
Table of Contents
- [Big Bang Overview](#big-bang-overview)
- [Key Concepts](#key-concepts)
- [Big Bang](#big-bang)
- [Big Bang Components](#big-bang-components)
- [Flux v2](#flux-v2)
- [GitOps](#gitops)
- [SOPS](#sops)
- [Kustomize](#kustomize)
- [Helm](#helm)
- [Packages](#packages)
- [Architecture](#architecture)
- [Configuration](#configuration)
- [Deployment](#deployment)
- [Diagram](#diagram)
[[_TOC_]]
---
......
# Big Bang Production
Table of Contents
- [Big Bang Production](#big-bang-production)
- [Production Deployment](#production-deployment)
[[_TOC_]]
## Production Deployment
The gatekeeper `values` section should resemble below when deploying to production.
```
```yaml
# OPA Gatekeeper
#
gatekeeper:
......@@ -41,7 +39,8 @@ To validate it was deployed correctly on your cluster run the following command:
You should only see `kube-system` under `excludedNamespaces` section.
Output:
```
```yaml
name: allowed-docker-registries
resourceVersion: "10390"
uid: b51b3887-3cf8-4495-b37e-fb8ef31755db
......@@ -61,5 +60,3 @@ spec:
- registry1.dso.mil
- registry.dso.mil
```
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment