diff --git a/Packages.md b/Packages.md
index 4068a931debdcf6b628fff4c1e7143ee1a474af7..e8c966268d8c5dd2fa4e1f2f6c1ec09b23853e97 100644
--- a/Packages.md
+++ b/Packages.md
@@ -1,6 +1,7 @@
# Packages
Columns:
+
* Logging - fluentbit configurations for standardized logging
* Telemetry - Integration with Prometheus and dedicated Grafana dashboards as appropriate
* Tracing - Insertion of Tracing data for application traffic
@@ -11,7 +12,7 @@ Columns:
## Core
| Package | Status | Logging | Telemetry | Tracing | Network Policies | mTLS | Behavior Detection |
-| ---- | --- | ---|---|---|---|---|---|
+| --- | --- | --- | --- | --- | --- | --- | --- |
| [Istio Operator](https://repo1.dso.mil/platform-one/big-bang/apps/core/istio-operator) |  | No | No | No | [Yes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/641) | Yes | No |
| [Istio Controlplane](https://repo1.dso.mil/platform-one/big-bang/apps/core/istio-controlplane) |  | No | No | No | [Yes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/632) | Yes | No |
| [Jaeger](https://repo1.dso.mil/platform-one/big-bang/apps/core/jaeger) |  | No | Yes | Yes | [Yes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/602) | [Yes (STRICT)](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/1295) | No |
@@ -28,6 +29,7 @@ Columns:
| [Promtail](https://repo1.dso.mil/platform-one/big-bang/apps/sandbox/promtail)  |  | No | No | No | Yes | [Yes (STRICT)](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/1516) | No |
| [Loki](https://repo1.dso.mil/platform-one/big-bang/apps/sandbox/loki)  |  | No | No | No | Yes | [Yes (STRICT)](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/1516) | No |
| [Tempo](https://repo1.dso.mil/platform-one/big-bang/apps/sandbox/tempo)  |  | No | Yes | Yes | [Yes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/1253) | [Yes (STRICT)](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/1762) | No |
+
<!-- | [NeuVector](https://repo1.dso.mil/platform-one/big-bang/apps/sandbox/neuvector)  |  | No | No | No | No | No | No | -->
## Supported Add-Ons
@@ -35,7 +37,7 @@ Columns:
### Security
| Package | Status | Logging | Telemetry | Tracing | Network Policies | mTLS | Behavior Detection |
-| ---- | --- | ---|---|---|---|---|---|
+| --- | --- | --- | --- | --- | --- | --- | --- |
| [Keycloak](https://repo1.dso.mil/platform-one/big-bang/apps/security-tools/keycloak) |  | No | No | No | [Yes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/536) | Yes (PERMISSIVE) | No |
| [Anchore Enterprise](https://repo1.dso.mil/platform-one/big-bang/apps/security-tools/anchore-enterprise) |  | No | No | No | [Yes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/505) | [Yes (STRICT)](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/1594) | No |
| [Authservice](https://repo1.dso.mil/platform-one/big-bang/apps/core/authservice) |  | No | Yes | Yes | [Yes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/511) | [Yes (STRICT)](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/1329) | No |
@@ -44,7 +46,7 @@ Columns:
### Development Tools
| Package | Status | Logging | Telemetry | Tracing | Network Policies | mTLS | Behavior Detection |
-| ---- | --- | ---|---|---|---|---|---|
+| --- | --- | --- | --- | --- | --- | --- | --- |
| [Gitlab](https://repo1.dso.mil/platform-one/big-bang/apps/developer-tools/gitlab) |  | No | No | No | [Yes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/504) | [Yes (STRICT)](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/1724) | No |
| [Gitlab Runner](https://repo1.dso.mil/platform-one/big-bang/apps/developer-tools/gitlab-runner) |  | No | No | No | [Yes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/522) | [Yes (STRICT)](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/1724)* | No |
| [Nexus](https://repo1.dso.mil/platform-one/big-bang/apps/developer-tools/nexus) |  | No | No | No | [Yes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/544) | [Yes (STRICT)](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/1605) | No |
@@ -55,21 +57,21 @@ Columns:
### Collaboration Tools
| Package | Status | Logging | Telemetry | Tracing | Network Policies | mTLS | Behavior Detection |
-| ---- | --- | ---|---|---|---|---|---|
+| --- | --- | --- | --- | --- | --- | --- | --- |
| [Mattermost](https://repo1.dso.mil/platform-one/big-bang/apps/collaboration-tools/mattermost) |  | No | No | No | [Yes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/515) | [No](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/issues/623) | No |
| [Mattermost Operator](https://repo1.dso.mil/platform-one/big-bang/apps/collaboration-tools/mattermost-operator) |  | No | No | No | [Yes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/499) | [Yes (STRICT)](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/1531) | No |
### Application Utilities
| Package | Status | Logging | Telemetry | Tracing | Network Policies | mTLS | Behavior Detection |
-| ---- | --- | ---|---|---|---|---|---|
+| --- | --- | --- | --- | --- | --- | --- | --- |
| [MinIO](https://repo1.dso.mil/platform-one/big-bang/apps/application-utilities/minio) |  | No | No | No | [Yes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/550) | [Yes (STRICT)](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/1566) | No |
| [MinIO Operator](https://repo1.dso.mil/platform-one/big-bang/apps/application-utilities/minio-operator) |  | No | No | No | [Yes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/685)] | [Yes (STRICT)](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/1554) |No |
### Cluster Utilities
| Package | Status | Logging | Telemetry | Tracing | Network Policies | mTLS | Behavior Detection |
-| ---- | --- | ---|---|---|---|---|---|
+| --- | --- | --- | --- | --- | --- | --- | --- |
| [Argocd](https://repo1.dso.mil/platform-one/big-bang/apps/core/argocd) | | No | No | No | [Yes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/572) | [Yes (STRICT)](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/1368) | No |
| [Velero](https://repo1.dso.mil/platform-one/big-bang/apps/cluster-utilities/velero) |  | No | No | No | [Yes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/552) | [Yes (STRICT)](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/1451) | No |
| [Metrics Server](https://repo1.dso.mil/platform-one/big-bang/apps/sandbox/metrics-server)  |  | No | No | No | [Yes](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/merge_requests/1738) | Yes (PERMISSIVE) | No |
diff --git a/chart/templates/kyverno/policies/values.yaml b/chart/templates/kyverno/policies/values.yaml
index ebbb76f59489f90d9a34055bcdd7a0cbae5eb76e..b0dca015b0a6d8fa899f83db36f42269f6b5c762 100644
--- a/chart/templates/kyverno/policies/values.yaml
+++ b/chart/templates/kyverno/policies/values.yaml
@@ -454,7 +454,7 @@ policies:
# NOTE: This restricts the ability to have PVCs when using a local path provisioner storage class (i.e. k3d default).
# To override either disable this policy (not ideal) or add an allowed wildcard matching where local paths are provisioned.
- # See `/docs/example_configs/policy-overrides-k3d.yaml` for an example of how to do this for k3d.
+ # See `docs/assets/configs/example/policy-overrides-k3d.yaml` for an example of how to do this for k3d.
restrict-host-path-mount-pv:
validationFailureAction: enforce
diff --git a/chart/values.yaml b/chart/values.yaml
index c2bc79708d6a5dad51ec91f2c71449e80f48dcb0..d9436a9e002475368508360f742f1d7d62f74db3 100644
--- a/chart/values.yaml
+++ b/chart/values.yaml
@@ -56,7 +56,7 @@ sso:
# -- Keycloak realm containing clients
realm: baby-yoda
- # -- Keycloak's certificate authority (PEM Format). Entered using chomp modifier (see ./docs/example_configs/dev-sso-values.yaml for example). Used by authservice to support SSO for various packages
+ # -- Keycloak's certificate authority (PEM Format). Entered using chomp modifier (see docs/assets/configs/example/dev-sso-values.yaml for example). Used by authservice to support SSO for various packages
certificate_authority: ""
# -- Keycloak realm's json web key output, obtained at https://<keycloak-server>/auth/realms/<realm>/protocol/openid-connect/certs
diff --git a/docs/README.md b/docs/README.md
index d5ff4ff3014c2d083ea2a7ccf95765b8a531444d..961b50d043e28dd41ff26427d8d3bdab946ae271 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -14,11 +14,11 @@
## How do I deploy BigBang?
-**Note:** The Deployment Process and Pre-Requisites will vary depending on the deployment scenario. The [Quick Start Demo Deployment](guides/deployment_scenarios/quickstart.md) for example, allows some steps to be skipped due to a mixture of automation and generically reusable demo configuration that satisfies pre-requisites.
-The following is a general overview of the process, the [deployment guides](guides/deployment_scenarios) go into more detail.
+**Note:** The Deployment Process and Pre-Requisites will vary depending on the deployment scenario. The [Quick Start Demo Deployment](guides/deployment-scenarios/quickstart.md) for example, allows some steps to be skipped due to a mixture of automation and generically reusable demo configuration that satisfies pre-requisites.
+The following is a general overview of the process, the [deployment guides](guides/deployment-scenarios) go into more detail.
1. Satisfy Pre-Requisites:
- * Provision a Kubernetes Cluster according to [best practices](guides/prerequisites/kubernetes_preconfiguration.md#best-practices).
+ * Provision a Kubernetes Cluster according to [best practices](guides/prerequisites/kubernetes-preconfiguration.md#best-practices).
* Ensure the Cluster has network connectivity to a Git Repo you control.
* Install Flux GitOps Operator on the Cluster.
* Configure Flux, the Cluster, and the Git Repo for GitOps Deployments that support deploying encrypted values.
diff --git a/docs/airgap/README.md b/docs/airgap/README.md
index 737eeb93fe217870bf038dc76831b4472815f176..89fe0a73d41fe0f470da38f0d2488a08af460342 100644
--- a/docs/airgap/README.md
+++ b/docs/airgap/README.md
@@ -188,7 +188,7 @@ docker load -i registry2.tar #on your registry server
docker load -i k3s.tar
```
-* Use the script [registry.sh](./scripts/registry.sh) to create registry;
+* Use the script [registry.sh](../assets/scripts/airgap-dev/registry.sh) to create registry;
```shell
$ chmod +x registry.sh && sudo ./registry.sh
@@ -338,7 +338,7 @@ kubectl create ns bigbang
Installing Big Bang in an air gap environment currently uses the Helm charts from the **[Big Bang Repo](https://repo1.dso.mil/platform-one/big-bang/bigbang)**.
-All changes are modified in the custom [values.yaml](./scripts/values.yaml) file. Modify as needed and replace IP.
+All changes are modified in the custom [values.yaml](../assets/scripts/airgap-dev/values.yaml) file. Modify as needed and replace IP.
Change the hostname for the installation. It is currently set to the development domain:
diff --git a/docs/airgap/developer/developer.md b/docs/airgap/k3d.md
similarity index 96%
rename from docs/airgap/developer/developer.md
rename to docs/airgap/k3d.md
index 68c0dbcd394500e4f18aba93f0a6c7e1d51c3885..f0cb0a498023cd9c3880e5884c59f5639bce674d 100644
--- a/docs/airgap/developer/developer.md
+++ b/docs/airgap/k3d.md
@@ -44,7 +44,7 @@ To test Airgap BigBang on k3d
$ k3d cluster create --image "rancher/k3s:v1.20.5-rc1-k3s1" --api-port "33989" -s 1 -a 2 -v "${HOME}/registries.yaml:/etc/rancher/k3s/registries.yaml" -v /etc/machine-id:/etc/machine-id -v "${HOME}/certs/host.k3d.internal.public.pem:/etc/ssl/certs/registry1.pem" --k3s-server-arg "--disable=traefik" --k3s-server-arg "--disable=metrics-server" --k3s-server-arg "--tls-san=$PRIVATEIP" -p 80:80@loadbalancer -p 443:443@loadbalancer
```
-- Block all egress with `iptables` except those going to instance IP before deploying bigbang by running [k3d_airgap.sh](./scripts/k3d_airgap.sh)
+- Block all egress with `iptables` except those going to instance IP before deploying bigbang by running [airgap.sh](../assets/scripts/airgap-dev/airgap.sh)
```shell
sudo ./k3d_airgap.sh
diff --git a/docs/airgap/developer/pipeline.md b/docs/airgap/pipeline.md
similarity index 100%
rename from docs/airgap/developer/pipeline.md
rename to docs/airgap/pipeline.md
diff --git a/docs/airgap/scripts/README.md b/docs/airgap/scripts/README.md
deleted file mode 100644
index 739e591a654e33bd5d32ea512a6d66c6bbbeea76..0000000000000000000000000000000000000000
--- a/docs/airgap/scripts/README.md
+++ /dev/null
@@ -1,3 +0,0 @@
-# README
-
-The directory contains all the utility scripts and examples.
diff --git a/docs/airgap/scripts/synker.md b/docs/airgap/synker.md
similarity index 100%
rename from docs/airgap/scripts/synker.md
rename to docs/airgap/synker.md
diff --git a/docs/example_configs/dev-sso-values.yaml b/docs/assets/configs/example/dev-sso-values.yaml
similarity index 99%
rename from docs/example_configs/dev-sso-values.yaml
rename to docs/assets/configs/example/dev-sso-values.yaml
index e382676cd07a42b0020ee17dc63c2159df29581b..c247b96ff015b9aca0d0ed8b607fe940908e8796 100644
--- a/docs/example_configs/dev-sso-values.yaml
+++ b/docs/assets/configs/example/dev-sso-values.yaml
@@ -106,7 +106,7 @@ sso:
-----END CERTIFICATE-----
# # LetsEncrypt certificate authority for keycloak.bigbang.dev
- # # Use this CA if you deployed Keycloak with *.bigbang.dev certificate using ./docs/example_configs/keycloak-dev-values.yaml
+ # # Use this CA if you deployed Keycloak with *.bigbang.dev certificate using docs/assets/configs/example/keycloak-dev-values.yaml
# certificate_authority: |
# -----BEGIN CERTIFICATE-----
# MIIFazCCA1OgAwIBAgIRAIIQz7DSQONZRGPgu2OCiwAwDQYJKoZIhvcNAQELBQAw
diff --git a/docs/example_configs/google-auth-example-values.yaml b/docs/assets/configs/example/google-auth-example-values.yaml
similarity index 100%
rename from docs/example_configs/google-auth-example-values.yaml
rename to docs/assets/configs/example/google-auth-example-values.yaml
diff --git a/docs/example_configs/keycloak-dev-values.yaml b/docs/assets/configs/example/keycloak-dev-values.yaml
similarity index 100%
rename from docs/example_configs/keycloak-dev-values.yaml
rename to docs/assets/configs/example/keycloak-dev-values.yaml
diff --git a/docs/example_configs/policy-overrides-k3d.yaml b/docs/assets/configs/example/policy-overrides-k3d.yaml
similarity index 100%
rename from docs/example_configs/policy-overrides-k3d.yaml
rename to docs/assets/configs/example/policy-overrides-k3d.yaml
diff --git a/docs/example_configs/public-ingressgateway-cert.yaml b/docs/assets/configs/example/public-ingressgateway-cert.yaml
similarity index 100%
rename from docs/example_configs/public-ingressgateway-cert.yaml
rename to docs/assets/configs/example/public-ingressgateway-cert.yaml
diff --git a/docs/example_configs/vault-production-values.yaml b/docs/assets/configs/example/vault-production-values.yaml
similarity index 100%
rename from docs/example_configs/vault-production-values.yaml
rename to docs/assets/configs/example/vault-production-values.yaml
diff --git a/docs/k8s-storage/benchmark.yaml b/docs/assets/configs/k8s-storage/benchmark.yaml
similarity index 100%
rename from docs/k8s-storage/benchmark.yaml
rename to docs/assets/configs/k8s-storage/benchmark.yaml
diff --git a/docs/k8s-storage/rwx_test.yaml b/docs/assets/configs/k8s-storage/rwx-test.yaml
similarity index 100%
rename from docs/k8s-storage/rwx_test.yaml
rename to docs/assets/configs/k8s-storage/rwx-test.yaml
diff --git a/docs/developer/diagrams/BB_gitlab_ci_diagram.drawio b/docs/assets/diagrams/developer/bb-gitlab-ci-diagram.drawio
similarity index 100%
rename from docs/developer/diagrams/BB_gitlab_ci_diagram.drawio
rename to docs/assets/diagrams/developer/bb-gitlab-ci-diagram.drawio
diff --git a/docs/developer/imgs/BB_pipelines.png b/docs/assets/imgs/developer/bb-pipelines.png
similarity index 100%
rename from docs/developer/imgs/BB_pipelines.png
rename to docs/assets/imgs/developer/bb-pipelines.png
diff --git a/docs/developer/imgs/Infra_test_pipelines.png b/docs/assets/imgs/developer/infra-test-pipelines.png
similarity index 100%
rename from docs/developer/imgs/Infra_test_pipelines.png
rename to docs/assets/imgs/developer/infra-test-pipelines.png
diff --git a/docs/developer/imgs/Package_pipeline.png b/docs/assets/imgs/developer/package-pipeline.png
similarity index 100%
rename from docs/developer/imgs/Package_pipeline.png
rename to docs/assets/imgs/developer/package-pipeline.png
diff --git a/docs/guides/backups-and-migrations/images/doom.png b/docs/assets/imgs/guides/doom.png
similarity index 100%
rename from docs/guides/backups-and-migrations/images/doom.png
rename to docs/assets/imgs/guides/doom.png
diff --git a/docs/guides/prerequisites/grafana-dashboard-manage.jpeg b/docs/assets/imgs/guides/grafana-dashboard-manage.jpeg
similarity index 100%
rename from docs/guides/prerequisites/grafana-dashboard-manage.jpeg
rename to docs/assets/imgs/guides/grafana-dashboard-manage.jpeg
diff --git a/docs/guides/prerequisites/grafana-dashboard.jpeg b/docs/assets/imgs/guides/grafana-dashboard.jpeg
similarity index 100%
rename from docs/guides/prerequisites/grafana-dashboard.jpeg
rename to docs/assets/imgs/guides/grafana-dashboard.jpeg
diff --git a/docs/guides/backups-and-migrations/images/volume-snapshot.png b/docs/assets/imgs/guides/volume-snapshot.png
similarity index 100%
rename from docs/guides/backups-and-migrations/images/volume-snapshot.png
rename to docs/assets/imgs/guides/volume-snapshot.png
diff --git a/docs/understanding_bigbang/images/kube-apiserver_webhooks_diagram.app.diagrams.net.png b/docs/assets/imgs/understanding-bigbang/kube-apiserver-webhooks-diagram.app.diagrams.net.png
similarity index 100%
rename from docs/understanding_bigbang/images/kube-apiserver_webhooks_diagram.app.diagrams.net.png
rename to docs/assets/imgs/understanding-bigbang/kube-apiserver-webhooks-diagram.app.diagrams.net.png
diff --git a/docs/understanding_bigbang/images/kube-apiserver_webhooks_diagram.app.diagrams.net.xml b/docs/assets/imgs/understanding-bigbang/kube-apiserver-webhooks-diagram.app.diagrams.net.xml
similarity index 100%
rename from docs/understanding_bigbang/images/kube-apiserver_webhooks_diagram.app.diagrams.net.xml
rename to docs/assets/imgs/understanding-bigbang/kube-apiserver-webhooks-diagram.app.diagrams.net.xml
diff --git a/docs/understanding_bigbang/images/logs_data_flow_diagram.app.diagrams.net.png b/docs/assets/imgs/understanding-bigbang/logs-data-flow-diagram.app.diagrams.net.png
similarity index 100%
rename from docs/understanding_bigbang/images/logs_data_flow_diagram.app.diagrams.net.png
rename to docs/assets/imgs/understanding-bigbang/logs-data-flow-diagram.app.diagrams.net.png
diff --git a/docs/understanding_bigbang/images/logs_data_flow_diagram.app.diagrams.net.xml b/docs/assets/imgs/understanding-bigbang/logs-data-flow-diagram.app.diagrams.net.xml
similarity index 100%
rename from docs/understanding_bigbang/images/logs_data_flow_diagram.app.diagrams.net.xml
rename to docs/assets/imgs/understanding-bigbang/logs-data-flow-diagram.app.diagrams.net.xml
diff --git a/docs/understanding_bigbang/images/metrics_data_flow_diagram.app.diagrams.net.png b/docs/assets/imgs/understanding-bigbang/metrics-data-flow-diagram.app.diagrams.net.png
similarity index 100%
rename from docs/understanding_bigbang/images/metrics_data_flow_diagram.app.diagrams.net.png
rename to docs/assets/imgs/understanding-bigbang/metrics-data-flow-diagram.app.diagrams.net.png
diff --git a/docs/understanding_bigbang/images/metrics_data_flow_diagram.app.diagrams.net.xml b/docs/assets/imgs/understanding-bigbang/metrics-data-flow-diagram.app.diagrams.net.xml
similarity index 100%
rename from docs/understanding_bigbang/images/metrics_data_flow_diagram.app.diagrams.net.xml
rename to docs/assets/imgs/understanding-bigbang/metrics-data-flow-diagram.app.diagrams.net.xml
diff --git a/docs/understanding_bigbang/images/network_encryption_and_ingress_diagram.app.diagrams.net.png b/docs/assets/imgs/understanding-bigbang/network-encryption-and-ingress-diagram.app.diagrams.net.png
similarity index 100%
rename from docs/understanding_bigbang/images/network_encryption_and_ingress_diagram.app.diagrams.net.png
rename to docs/assets/imgs/understanding-bigbang/network-encryption-and-ingress-diagram.app.diagrams.net.png
diff --git a/docs/understanding_bigbang/images/network_encryption_and_ingress_diagram.app.diagrams.net.xml b/docs/assets/imgs/understanding-bigbang/network-encryption-and-ingress-diagram.app.diagrams.net.xml
similarity index 100%
rename from docs/understanding_bigbang/images/network_encryption_and_ingress_diagram.app.diagrams.net.xml
rename to docs/assets/imgs/understanding-bigbang/network-encryption-and-ingress-diagram.app.diagrams.net.xml
diff --git a/docs/airgap/scripts/airgap.sh b/docs/assets/scripts/airgap-dev/airgap.sh
similarity index 100%
rename from docs/airgap/scripts/airgap.sh
rename to docs/assets/scripts/airgap-dev/airgap.sh
diff --git a/docs/airgap/scripts/config.toml.tmpl b/docs/assets/scripts/airgap-dev/config.toml.tmpl
similarity index 100%
rename from docs/airgap/scripts/config.toml.tmpl
rename to docs/assets/scripts/airgap-dev/config.toml.tmpl
diff --git a/docs/airgap/scripts/copy-containerd-config.yaml b/docs/assets/scripts/airgap-dev/copy-containerd-config.yaml
similarity index 100%
rename from docs/airgap/scripts/copy-containerd-config.yaml
rename to docs/assets/scripts/airgap-dev/copy-containerd-config.yaml
diff --git a/docs/airgap/scripts/deploy-images.sh b/docs/assets/scripts/airgap-dev/deploy-images.sh
similarity index 100%
rename from docs/airgap/scripts/deploy-images.sh
rename to docs/assets/scripts/airgap-dev/deploy-images.sh
diff --git a/docs/airgap/scripts/images.txt b/docs/assets/scripts/airgap-dev/images.txt
similarity index 100%
rename from docs/airgap/scripts/images.txt
rename to docs/assets/scripts/airgap-dev/images.txt
diff --git a/docs/airgap/scripts/package_images.sh b/docs/assets/scripts/airgap-dev/package-images.sh
similarity index 100%
rename from docs/airgap/scripts/package_images.sh
rename to docs/assets/scripts/airgap-dev/package-images.sh
diff --git a/docs/airgap/scripts/package_repos.sh b/docs/assets/scripts/airgap-dev/package-repos.sh
similarity index 100%
rename from docs/airgap/scripts/package_repos.sh
rename to docs/assets/scripts/airgap-dev/package-repos.sh
diff --git a/docs/airgap/scripts/registry_config.yaml b/docs/assets/scripts/airgap-dev/registry-config.yaml
similarity index 100%
rename from docs/airgap/scripts/registry_config.yaml
rename to docs/assets/scripts/airgap-dev/registry-config.yaml
diff --git a/docs/airgap/scripts/registry.sh b/docs/assets/scripts/airgap-dev/registry.sh
similarity index 100%
rename from docs/airgap/scripts/registry.sh
rename to docs/assets/scripts/airgap-dev/registry.sh
diff --git a/docs/airgap/scripts/values.yaml b/docs/assets/scripts/airgap-dev/values.yaml
similarity index 100%
rename from docs/airgap/scripts/values.yaml
rename to docs/assets/scripts/airgap-dev/values.yaml
diff --git a/docs/developer/scripts/k3d-dev.sh b/docs/assets/scripts/developer/k3d-dev.sh
similarity index 100%
rename from docs/developer/scripts/k3d-dev.sh
rename to docs/assets/scripts/developer/k3d-dev.sh
diff --git a/docs/airgap/developer/terraform/README.md b/docs/assets/terraform/airgap-dev/README.md
similarity index 100%
rename from docs/airgap/developer/terraform/README.md
rename to docs/assets/terraform/airgap-dev/README.md
diff --git a/docs/airgap/developer/terraform/main.tf b/docs/assets/terraform/airgap-dev/main.tf
similarity index 100%
rename from docs/airgap/developer/terraform/main.tf
rename to docs/assets/terraform/airgap-dev/main.tf
diff --git a/docs/airgap/developer/terraform/variables.tf b/docs/assets/terraform/airgap-dev/variables.tf
similarity index 100%
rename from docs/airgap/developer/terraform/variables.tf
rename to docs/assets/terraform/airgap-dev/variables.tf
diff --git a/docs/configuration.md b/docs/configuration.md
index ac661c1e49c6684274254df29a3bf5f72c10bb37..91ddd8964a585da7d4a5fa4d5ae3d745d9392338 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -47,21 +47,22 @@ The Big Bang [Environment Template](https://repo1.dso.mil/platform-one/big-bang/
Hostname is used to override the domain of deployed packages. This allows you to go to the DNS name of a server using the domain. For example, if the domain is `bigbang.dev`, Kiali can be reached at `kiali.bigbang.dev`.
-| Key | Description | Type | Default |
-|--|--|--|--|
-| `hostname` | Domain to use for deployed servers | Domain Name | `bigbang.dev`
+| Key | Description | Type | Default |
+| ---------- | ---------------------------------- | ----------- | ------------- |
+| `hostname` | Domain to use for deployed servers | Domain Name | `bigbang.dev` |
### `registryCredentials`
Registry credentials are used to pull images for Big Bang. By default, it points to Iron Bank, but can be modified to use a private registry. These credentials are passed down to all relevant namespaces as an image pull secret.
-| Key | Description | Type | Default |
-|--|--|--|--|
-| `registry` | Container registry location | Domain Name | `registry1.dso.mil`
-| `username`* | Container registry username | String | "" |
-| `password`* | User's password | String | "" |
-| `email` | User's email | Email | "" |
-> *Credentials should be SOPS encrypted
+| Key | Description | Type | Default |
+| ------------ | --------------------------- | ----------- | ------------------- |
+| `registry` | Container registry location | Domain Name | `registry1.dso.mil` |
+| `username`\* | Container registry username | String | "" |
+| `password`\* | User's password | String | "" |
+| `email` | User's email | Email | "" |
+
+> \*Credentials should be SOPS encrypted
### `flux`
@@ -69,7 +70,7 @@ Flux settings are used to setup the default continuous deployment configuration
| Key | Description | Type | Default |
|--|--|--|--|
-| `interval` | Polling interval to check for Git or Helm chart updates | ##m##s | 2m |
+| `interval` | Polling interval to check for Git or Helm chart updates | ##m##s (ex. 5m30s) | 2m |
| `install.retries` | The number of retries that should be attempted on Helm chart installation failures before bailing. | int | 3 |
| `upgrade.retries` | The number of retries that should be attempted on Helm chart upgrade failures before bailing. | int | 3 |
| `rollback.timeout` | The time to wait for any individual Kubernetes operation (like Jobs for hooks) during the performance of a Helm rollback action. | ##m##s | 5m |
@@ -97,12 +98,12 @@ Each package (e.g. `istio`, `clusterAuditor`) has configuration to control how B
Big Bang deploys four flux resources that can be customized:
-| Resource | Controls | Location |
-|--|--|--|
-| GitRepository | Environment | Top-level manifest (e.g. `dev.yaml`, `prod.yaml`)
-| Kustomization | Environment | Top-level manifest (e.g. `dev.yaml`, `prod.yaml`)
-| GitRepository | Big Bang | [Link](../base/gitrepository.yaml) |
-| HelmRelease | Big Bang | [Link](../base/helmrelease.yaml) |
+| Resource | Controls | Location |
+| ------------- | ----------- | ------------------------------------------------- |
+| GitRepository | Environment | Top-level manifest (e.g. `dev.yaml`, `prod.yaml`) |
+| Kustomization | Environment | Top-level manifest (e.g. `dev.yaml`, `prod.yaml`) |
+| GitRepository | Big Bang | [Link](../base/gitrepository.yaml) |
+| HelmRelease | Big Bang | [Link](../base/helmrelease.yaml) |
In addition, each package contains its own GitRepository and HelmRelease resource that can be customized. Look in the [Helm chart templates](../chart/templates) for the these resources.
@@ -111,7 +112,7 @@ Settings for any of these resources can be overridden by [patching](https://kube
- Updating flux-system component resource usage
- [Example `kustomization.yaml`](https://repo1.dso.mil/platform-one/big-bang/customers/template/-/tree/main#adjust-resource-allocation-for-a-flux-system-component)
- This patch could be used to adjust the resources requested by the `flux-system/helm-controller` resource. A similar patch could be used to adjust the resources required by the other flux components.
- > NOTE: If flux is under-resourced, occasionally requests can fail in a manner that looks like a network connectivity issue (use with caution)
+ > NOTE: If flux is under-resourced, occasionally requests can fail in a manner that looks like a network connectivity issue (use with caution)
- Adding environment variables to flux-system components
- [Example `kustomization.yaml`](https://repo1.dso.mil/platform-one/big-bang/customers/template/-/tree/main#adjust-resource-allocation-for-a-flux-system-component)
- This patch could be used to add AWS credential environment variables into the `flux-system/kustomize-controller` resource to enable SOPS decryption using a KMS key from outside of AWS.
@@ -127,17 +128,17 @@ In your `kustomization.yaml` under your environment, here is an example of how t
```yaml
bases:
-- https://repo1.dso.mil/platform-one/big-bang/bigbang.git/base/?ref=v1.2.*
+ - https://repo1.dso.mil/platform-one/big-bang/bigbang.git/base/?ref=v1.2.*
patchesStrategicMerge:
-- |-
- apiVersion: source.toolkit.fluxcd.io/v1beta2
- kind: GitRepository
- metadata:
- name: bigbang
- spec:
- ref:
- $patch: replace
- semver: "1.2.x"
+ - |-
+ apiVersion: source.toolkit.fluxcd.io/v1beta2
+ kind: GitRepository
+ metadata:
+ name: bigbang
+ spec:
+ ref:
+ $patch: replace
+ semver: "1.2.x"
```
> Note: You must put the version in two places, once for Kustomize to pull the right configuration base and two for Git Repository to pull the right Helm Chart.
@@ -186,12 +187,12 @@ If you have pull credentials for your docker registry, add them to `secrets.enc.
apiVersion: v1
kind: Secret
metadata:
- name: common-bb
+ name: common-bb
stringData:
- values.yaml: |-
- registryCredentials:
- username: iron-bank-user
- password: iron-bank-password
+ values.yaml: |-
+ registryCredentials:
+ username: iron-bank-user
+ password: iron-bank-password
```
You will also need to update your `kustomization.yaml` to merge with the existing secret:
@@ -199,12 +200,12 @@ You will also need to update your `kustomization.yaml` to merge with the existin
```yaml
namespace: bigbang
patchesStrategicMerge:
-- secrets.enc.yaml
+ - secrets.enc.yaml
```
## Package settings
-Besides the [global settings](#big-bang-globals), package settings are defined by the individual packet's helm charts. You can find these by reviewing the `git.registry` setting for the package in Big Bang's [default values.yaml](../chart/values.yaml)
+Besides the [global settings](#big-bang-globals), package settings are defined by the individual packet's helm charts. You can find these by reviewing the `git.registry` setting for the package in Big Bang's [default values.yaml](../chart/values.yaml).
To modify a non-sensitive package setting, add it to your `configmap.yaml`. For sensitive information, follow the pattern for setting [registry pull credentials](#registry-pull-credentials). Here we disable `twistlock` and set `gatekeeper`'s replicas to `1`:
diff --git a/docs/developer/README.md b/docs/developer/README.md
index 891c57ee949d7a97ddadcac9403a55538f7d2f8f..aa0a693c68ff39b8eb9effe4dd697e06b9b8dc68 100644
--- a/docs/developer/README.md
+++ b/docs/developer/README.md
@@ -27,7 +27,7 @@ Big Bang is a helm chart of helm charts. Big Bang uses [Flux 2](https://fluxcd.i
## Add Package to Big Bang
-[Integrate Package with Big Bang](./package-integration.md)
+[Integrate Package with Big Bang](./package-integration/README.md)
## Big Bang Code Through Party Bus Pipeline
diff --git a/docs/developer/scripts/README.md b/docs/developer/aws-k3d-script.md
similarity index 66%
rename from docs/developer/scripts/README.md
rename to docs/developer/aws-k3d-script.md
index 9963dac49ddf6d50157782e495d3cb59efd11266..5a105adfba9f9d02134e0ba8cee64f25a2153da2 100644
--- a/docs/developer/scripts/README.md
+++ b/docs/developer/aws-k3d-script.md
@@ -8,32 +8,32 @@ The instance will automatically terminate in the middle of the night at 08:00 UT
1. Install aws cli
- ```shell
- curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
- # sudo apt install unzip -y
- unzip awscliv2.zip
- sudo ./aws/install
- rm -rf aws
- rm awscliv2.zip
- aws --version
- ```
+ ```shell
+ curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
+ # sudo apt install unzip -y
+ unzip awscliv2.zip
+ sudo ./aws/install
+ rm -rf aws
+ rm awscliv2.zip
+ aws --version
+ ```
1. Configure aws cli
- ```shell
- aws configure
- # aws_access_key_id - The AWS access key part of your credentials
- # aws_secret_access_key - The AWS secret access key part of your credentials
- # region - us-gov-west-1
- # output - JSON
+ ```shell
+ aws configure
+ # aws_access_key_id - The AWS access key part of your credentials
+ # aws_secret_access_key - The AWS secret access key part of your credentials
+ # region - us-gov-west-1
+ # output - JSON
- # Verify configuration
- aws configure list
- ```
+ # Verify configuration
+ aws configure list
+ ```
1. Install jq
- Follow jq installation instructions for your workstation operating system.
- <https://stedolan.github.io/jq/download/>
+ Follow jq installation instructions for your workstation operating system.
+ <https://stedolan.github.io/jq/download/>
1. Mac users will need to install the GNU version of the sed command.
<https://medium.com/@bramblexu/install-gnu-sed-on-mac-os-and-set-it-as-default-7c17ef1b8f64>
@@ -42,8 +42,8 @@ The instance will automatically terminate in the middle of the night at 08:00 UT
The default with no options specified is to use the EC2 public IP for the k3d cluster and the security group.
-```console
-./docs/developer/scripts/k3d-dev.sh -h
+```shell
+./docs/assets/scripts/developer/k3d-dev.sh -h
AWS User Name: your.name
Usage:
k3d-dev.sh -b -p -m -d -h
@@ -71,13 +71,13 @@ The Big Bang product is tightly coupled with the GitOps tool FluxCD. Before you
1. If you are on a Mac insure that you have GNU sed command installed. Otherwise you will see this error and the kubeconfig will not be updated with the IP from the instance.
- ```console
- copy kubeconfig
- config 100% 3019 72.9KB/s 00:00
- sed: 1: "...": extra characters at the end of p command
+ ```console
+ copy kubeconfig
+ config 100% 3019 72.9KB/s 00:00
+ sed: 1: "...": extra characters at the end of p command
- ```
+ ```
2. If you get a failure from the script study and correct the error. Then run script with "-d" option to clean up resources. Then re-run your original command.
-3. Occasionally a ssh command will fail because of connection problems. If this happens the script will fail with "unexpected EOF". Simply try again. Run the script with ```-d``` to clean up resources. Then re-run your original command.
+3. Occasionally a ssh command will fail because of connection problems. If this happens the script will fail with "unexpected EOF". Simply try again. Run the script with `-d` to clean up resources. Then re-run your original command.
diff --git a/docs/developer/ci-workflow.md b/docs/developer/ci-workflow.md
index 15217915522f54fcaa3cb4c88d0e260bb6cc64f9..03d75f0d5d6c1da06218624129f4c7bfffb7ae82 100644
--- a/docs/developer/ci-workflow.md
+++ b/docs/developer/ci-workflow.md
@@ -13,9 +13,9 @@ This pipeline is triggered by the following for individual bigbang packages:
- manual tag events
- commits to default branch
-
+
-[Link to draw.io diagram file](diagrams/BB_gitlab_ci_diagram.drawio). This diagram file should be modified on draw.io and exported into this repository when the developer / ci workflow changes. It is provided here for ease of use.
+[Link to draw.io diagram file](../assets/diagrams/developer/bb-gitlab-ci-diagram.drawio). This diagram file should be modified on draw.io and exported into this repository when the developer / ci workflow changes. It is provided here for ease of use.
### Configuration Validation
@@ -45,9 +45,9 @@ This pipeline is triggered by the following for individual bigbang packages:
The pipeline is split into several stages:
-
+
-[Link to draw.io diagram file](diagrams/BB_gitlab_ci_diagram.drawio). This diagram file should be modified on draw.io and exported into this repository when the developer / ci workflow changes. It is provided here for ease of use.
+[Link to draw.io diagram file](../assets/diagrams/developer/bb-gitlab-ci-diagram.drawio). This diagram file should be modified on draw.io and exported into this repository when the developer / ci workflow changes. It is provided here for ease of use.
### Pre Vars
@@ -99,9 +99,9 @@ The infrastructure pipeline is designed to have _no_ human interaction, and is s
More information on the full set of infrastructure tests are below:
-
+
-[Link to draw.io diagram file](diagrams/BB_gitlab_ci_diagram.drawio). This diagram file should be modified on draw.io and exported into this repository when the developer / ci workflow changes. It is provided here for ease of use.
+[Link to draw.io diagram file](../assets/diagrams/developer/bb-gitlab-ci-diagram.drawio). This diagram file should be modified on draw.io and exported into this repository when the developer / ci workflow changes. It is provided here for ease of use.
### Network Creation
@@ -121,7 +121,7 @@ It is a hard requirement at this stage that every cluster outputs an admin scope
Given the kubeconfig created in the previous stage, BigBang is installed on the cluster using the same installation process used in the smoke tests.
-Like any BigBang installation, several cluster requirements (see [Pre-requisites](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/tree/master/docs/guides/prerequisites)) must be met before BigBang is installed, and it is up to the vendor to ensure those requirements are met.
+Like any BigBang installation, several cluster requirements (see [Pre-requisites](../guides/prerequisites/README.md)) must be met before BigBang is installed, and it is up to the vendor to ensure those requirements are met.
### Big Bang Tests
diff --git a/docs/developer/development-environment.md b/docs/developer/development-environment.md
index a0b5a74b6894eb5a5c82b6a3c75e713129219233..eac4b57aa6a221592c4b647a913decc3933dbbc3 100644
--- a/docs/developer/development-environment.md
+++ b/docs/developer/development-environment.md
@@ -24,5 +24,4 @@ There is a script [/docs/developer/scripts/k3d-dev.sh](./scripts/) that automate
- [jq](https://stedolan.github.io/jq/download/)
- optional: [kustomize](https://kubectl.docs.kubernetes.io/installation/kustomize/)
-> For additional installation details, see [Software Installation and Verification Commands to run from Bash](https://repo1.dso.mil/platform-one/onboarding/big-bang/engineering-cohort/-/blob/master/lab_guides/01-Preflight-Access-Checks/A-software-check.md)
-
+> For additional installation details, see [Software Installation and Verification Commands to run from Bash](https://repo1.dso.mil/platform-one/onboarding/big-bang/engineering-cohort/-/blob/master/lab_guides/01-Preflight-Access-Checks/A-software-check.md)
\ No newline at end of file
diff --git a/docs/developer/package-integration.md b/docs/developer/package-integration.md
deleted file mode 100644
index a15ed8dd5f5591e22d362cff722630347187354c..0000000000000000000000000000000000000000
--- a/docs/developer/package-integration.md
+++ /dev/null
@@ -1,17 +0,0 @@
-# Big Bang Integration: Overview
-
-The following documents should be followed, in order, to fully integrate a new package into Big Bang:
-
-1. [Upstream Helm Chart](./package-integration/package-integration-upstream.md): Initialize package workspace using an upstream Helm chart
-1. [CICD Pipeline](./package-integration/package-integration-pipeline.md): Establish a baseline package pipeline for testing changes
-1. [Flux Helm Chart](./package-integration/package-integration-flux.md): Create Flux compatible GitOps Helm chart required by Big Bang
-1. [Service mesh](./package-integration/package-integration-service-mesh.md): Integrate with service mesh for ingress/egress
-1. [Monitoring](./package-integration/package-integration-monitoring.md): Enable metrics scraping on product
-1. [Database](./package-integration/package-integration-database.md): If required, add internal and external database support using Big Bang values
-1. [Object Storage](./package-integration/package-integration-storage.md): If required, add internal or external object storage support using Big Bang values
-1. [Single-sign On](./package-integration/package-integration-sso.md): If available, add single-sign on (SSO) through internal or external identify provider.
-1. [Additional Tests](./package-integration/package-integration-testing.md): Add testing to validate basic functionality
-1. [Network Policies](./package-integration/package-integration-network-policies.md): Add ingress/egress policies to restrict network traffic for security
-1. [Policy Enforcement](./package-integration/package-integration-policy-enforcement.md): Update package to comply with default security and governance policies in Big Bang
-1. [Supported Package](./package-integration/product-integration-supported.md): Migrate package into the Big Bang repo as a supported package
-1. [Final Documentation](./package-integration/package-integration-documentation.md): Add additional Big Bang documentation for final release
diff --git a/docs/developer/package-integration/README.md b/docs/developer/package-integration/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..65b33c8b7aa7d0f512c5e174b7fe816faf65fdee
--- /dev/null
+++ b/docs/developer/package-integration/README.md
@@ -0,0 +1,17 @@
+# Big Bang Integration: Overview
+
+The following documents should be followed, in order, to fully integrate a new package into Big Bang:
+
+1. [Upstream Helm Chart](upstream.md): Initialize package workspace using an upstream Helm chart
+1. [CICD Pipeline](pipeline.md): Establish a baseline package pipeline for testing changes
+1. [Flux Helm Chart](flux.md): Create Flux compatible GitOps Helm chart required by Big Bang
+1. [Service mesh](service-mesh.md): Integrate with service mesh for ingress/egress
+1. [Monitoring](monitoring.md): Enable metrics scraping on product
+1. [Database](database.md): If required, add internal and external database support using Big Bang values
+1. [Object Storage](storage.md): If required, add internal or external object storage support using Big Bang values
+1. [Single-sign On](sso.md): If available, add single-sign on (SSO) through internal or external identify provider.
+1. [Additional Tests](testing.md): Add testing to validate basic functionality
+1. [Network Policies](network-policies.md): Add ingress/egress policies to restrict network traffic for security
+1. [Policy Enforcement](policy-enforcement.md): Update package to comply with default security and governance policies in Big Bang
+2. [Supported Package](supported.md): Migrate package into the Big Bang repo as a supported package
+3. [Final Documentation](documentation.md): Add additional Big Bang documentation for final release
diff --git a/docs/developer/package-integration/package-integration-database.md b/docs/developer/package-integration/database.md
similarity index 100%
rename from docs/developer/package-integration/package-integration-database.md
rename to docs/developer/package-integration/database.md
diff --git a/docs/developer/package-integration/package-integration-documentation.md b/docs/developer/package-integration/documentation.md
similarity index 79%
rename from docs/developer/package-integration/package-integration-documentation.md
rename to docs/developer/package-integration/documentation.md
index 6a4217a39716d495389180008ba6d8126b11c0c6..589866230122b8528b34afaad50a2798d4963b5f 100644
--- a/docs/developer/package-integration/package-integration-documentation.md
+++ b/docs/developer/package-integration/documentation.md
@@ -4,6 +4,6 @@ Big Bang requires some additional documentation for supported packages to help u
- Package Architecture: See [Big Bang's Architecture instructions](../../../charter/packages/ref-package/Architecture.md). Examples are included in [charter/packages](../../../charter/packages)
- [Big Bang Packages](../../../charter/BigBangPackages.md)
-- [Default Credentials](../../guides/using_bigbang/default_credentials.md)
-- [Licensing](../../understanding_bigbang/licensing_expectations.md)
-- [Minimum Hardware Requirements](../../guides/prerequisites/minimum_hardware_requirements.md)
+- [Default Credentials](../../guides/using-bigbang/default-credentials.md)
+- [Licensing](../../understanding-bigbang/licensing-expectations.md)
+- [Minimum Hardware Requirements](../../guides/prerequisites/minimum-hardware-requirements.md)
diff --git a/docs/developer/package-integration/package-integration-flux.md b/docs/developer/package-integration/flux.md
similarity index 95%
rename from docs/developer/package-integration/package-integration-flux.md
rename to docs/developer/package-integration/flux.md
index efe4f1e4ea585d1a60c73c3984175fbb0d919356..6f1967fa6e7b79bd7a86c812eef330f242a09db7 100644
--- a/docs/developer/package-integration/package-integration-flux.md
+++ b/docs/developer/package-integration/flux.md
@@ -12,7 +12,7 @@ Big Bang uses a continuous deployment tool, [Flux](https://fluxcd.io/) to deploy
- [Kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl)
- [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
- A multi-node Kubernetes cluster to deploy Big Bang and your package
-- A [Big Bang project containing the upstream Helm chart](./package-integration-upstream.md)
+- A [Big Bang project containing the upstream Helm chart](./upstream.md)
> Throughout this document, we will be setting up an application called `podinfo` as a demonstration
@@ -218,12 +218,12 @@ Package values (chart/values.yaml) should contain upstream values plus any place
Big Bang has a few options for overwriting values in packages. The package's `HelmRelease`, that we created earlier, contains a `ValuesFrom` section that references a secret with `common`, `default`, and `overlay` keys. Each of these keys can contain a set of override values that get passed down to the package. Here is a table explaining the difference between the possible overlays:
-|Name|Description|Source|Priority|
-|--|--|--|--|
-| `overlay` | Values provided by user when deploying Big Bang | `bigbang/values.yaml`:`$PKGNAME.values.*` | Highest 1 |
-| `default` | Values created by Big Bang | `bigbang/templates/$PKGNAME/values.yaml`:`*` | 2 |
-| `common` | Big Bang values common to all packages | Not currently used | 3 |
-| `package` | Package defaults | `chart/values.yaml`:`*` | Lowest 4 |
+| Name | Description | Source | Priority |
+| --------- | ----------------------------------------------- | -------------------------------------------- | --------- |
+| `overlay` | Values provided by user when deploying Big Bang | `bigbang/values.yaml`:`$PKGNAME.values.*` | Highest 1 |
+| `default` | Values created by Big Bang | `bigbang/templates/$PKGNAME/values.yaml`:`*` | 2 |
+| `common` | Big Bang values common to all packages | Not currently used | 3 |
+| `package` | Package defaults | `chart/values.yaml`:`*` | Lowest 4 |
This means that if a user provides a value for the package, that overwrites the value Big Bang or the package would create.
diff --git a/docs/developer/package-integration/package-integration-monitoring.md b/docs/developer/package-integration/monitoring.md
similarity index 100%
rename from docs/developer/package-integration/package-integration-monitoring.md
rename to docs/developer/package-integration/monitoring.md
diff --git a/docs/developer/package-integration/package-integration-network-policies.md b/docs/developer/package-integration/network-policies.md
similarity index 99%
rename from docs/developer/package-integration/package-integration-network-policies.md
rename to docs/developer/package-integration/network-policies.md
index 6f0023c814ce728f651043050784047c847ba83f..80a131843a0d58c3f568a612cee0f457ed0ea6f4 100644
--- a/docs/developer/package-integration/package-integration-network-policies.md
+++ b/docs/developer/package-integration/network-policies.md
@@ -65,7 +65,7 @@ spec:
### Was Something Important Blocked?
There are a few ways to determine if a network policy is blocking egress or ingress to or from a pod.
-- Test things from the pod's perspective using ssh/exec. See [this portion](../../guides/deployment_scenarios/sso_quickstart.md#step-18-update-inner-cluster-dns-on-the-workload-cluster) of the keycloak quickstart for an example of how do to that.
+- Test things from the pod's perspective using ssh/exec. See [this portion](../../guides/deployment-scenarios/sso-quickstart.md#step-18-update-inner-cluster-dns-on-the-workload-cluster) of the keycloak quickstart for an example of how do to that.
- Curl a pod's IP from another pod to see if network polices are blocking that traffic. Use `kubectl pod -o wide -n <podNamespace>` to see pod IP addresses.
- Check the pod logs (or curl from one container to the service) for a `context deadline exceeded` or `connection refused` message.
diff --git a/docs/developer/package-integration/package-integration-pipeline.md b/docs/developer/package-integration/pipeline.md
similarity index 95%
rename from docs/developer/package-integration/package-integration-pipeline.md
rename to docs/developer/package-integration/pipeline.md
index 62a1136342a7aa7258356215f8ee4763cc263bdf..731bbfb1370754824b737be6a69640b7bb8bf0ee 100644
--- a/docs/developer/package-integration/package-integration-pipeline.md
+++ b/docs/developer/package-integration/pipeline.md
@@ -6,7 +6,7 @@ Big Bang contains and uses a continuous deployment tool to deploy packages using
- [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
- [Docker CLI](https://docs.docker.com/get-docker/)
-- [Big Bang package project containing your Helm chart](./package-integration-upstream.md)
+- [Big Bang package project containing your Helm chart](./upstream.md)
> You will need to have the Container Registry enabled.This can be requested from the Big Bang team.
> Throughout this document, we will be setting up an application called `podinfo` as a demonstration.
@@ -105,4 +105,4 @@ Examples of components that contribute to a package being "Big Bang compatible":
This stage also allows any Big Bang Core or Addon packages to be deployed alongside a third-party or sandbox package for testing compatibility/functionality.
-To set this up in a package repo, see the guide [here](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/blob/master/docs/developer/package-integration/package-integration-flux.md).
+To set this up in a package repo, see the guide [here](./flux.md).
diff --git a/docs/developer/package-integration/package-integration-policy-enforcement.md b/docs/developer/package-integration/policy-enforcement.md
similarity index 100%
rename from docs/developer/package-integration/package-integration-policy-enforcement.md
rename to docs/developer/package-integration/policy-enforcement.md
diff --git a/docs/developer/package-integration/package-integration-service-mesh.md b/docs/developer/package-integration/service-mesh.md
similarity index 99%
rename from docs/developer/package-integration/package-integration-service-mesh.md
rename to docs/developer/package-integration/service-mesh.md
index 8ed3f8e6ca0dccfd9f81038aa41563d11d345b99..0d57f700f6adf929ac3d53b5b8ff8cda636ec193 100644
--- a/docs/developer/package-integration/package-integration-service-mesh.md
+++ b/docs/developer/package-integration/service-mesh.md
@@ -9,7 +9,7 @@
- [Kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl)
- [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
- A multi-node Kubernetes cluster to deploy Big Bang and your package
-- A [Big Bang project containing the upstream Helm chart](./package-integration-upstream.md)
+- A [Big Bang project containing the upstream Helm chart](./upstream.md)
> Throughout this document, we will be setting up an application called `podinfo` as a demonstration
diff --git a/docs/developer/package-integration/package-integration-sso.md b/docs/developer/package-integration/sso.md
similarity index 96%
rename from docs/developer/package-integration/package-integration-sso.md
rename to docs/developer/package-integration/sso.md
index e3cd2380d9cf39c0f3d93214ae5248bed42ed395..4826ba50630589ba380ae8958bc94e1e96ee7fa2 100644
--- a/docs/developer/package-integration/package-integration-sso.md
+++ b/docs/developer/package-integration/sso.md
@@ -5,8 +5,8 @@ Big Bang has configuration for Single Sign-On (SSO) authentication using an iden
## Prerequisites
The development environment can be set up in one of two ways:
-1. Two k3d clusters with keycloak in one cluster and Big Bang and all other apps in the second cluster (see [this quick start guide](../../guides/deployment_scenarios/sso_quickstart.md) for more information)
-2. One k3d cluster using metallb to have keycloak, Big Bang, and all other apps in the one cluster (see [this example config](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/blob/master/docs/example_configs/keycloak-dev-values.yaml) for more information)
+1. Two k3d clusters with keycloak in one cluster and Big Bang and all other apps in the second cluster (see [this quick start guide](../../guides/deployment-scenarios/sso-quickstart.md) for more information)
+2. One k3d cluster using metallb to have keycloak, Big Bang, and all other apps in the one cluster (see [this example config](../../assets/configs/example/keycloak-dev-values.yaml) for more information)
## Integration
diff --git a/docs/developer/package-integration/package-integration-storage.md b/docs/developer/package-integration/storage.md
similarity index 100%
rename from docs/developer/package-integration/package-integration-storage.md
rename to docs/developer/package-integration/storage.md
diff --git a/docs/developer/package-integration/product-integration-supported.md b/docs/developer/package-integration/supported.md
similarity index 98%
rename from docs/developer/package-integration/product-integration-supported.md
rename to docs/developer/package-integration/supported.md
index 912e478e2fdec4bbd6fbd0b99d56390e72700fc2..f3b5b28ae14b3d43dbf1715ff80ff3f701aa2768 100644
--- a/docs/developer/package-integration/product-integration-supported.md
+++ b/docs/developer/package-integration/supported.md
@@ -26,7 +26,7 @@ After [graduating your package](https://repo1.dso.mil/platform-one/bbtoc/-/tree/
1. Make a branch from the BigBang chart repository `master` branch. You can automatically create a branch from the Repo1 Gitlab issue. Or, in some cases you might manually create the branch. Name the branch with your issue number. For example, if your issue number is `9999` then your branch name can be `9999-my-description`. It is best practice to make branch names short and simple.
-1. Make sure the files described in this [document](./package-integration-flux.md) have been generated in `chart/templates/<your-package-name>` directory
+1. Make sure the files described in this [document](./flux.md) have been generated in `chart/templates/<your-package-name>` directory
1. More details about secret-*.yaml: The secret template is where the code for secrets go. Typically you will see secrets for imagePullSecret, sso, database, and possibly object storage. These secrets are a BigBang chart enhancement. They are created conditionally based on what the user enables in the config. For example if the app supports SSO and will need a Certificate Authority supplied to trust the connection to the IdP there should be a `secret-ca.yaml` template to populate a secret with the `sso.certificate_authority` value in the application namespace.
diff --git a/docs/developer/package-integration/package-integration-testing.md b/docs/developer/package-integration/testing.md
similarity index 98%
rename from docs/developer/package-integration/package-integration-testing.md
rename to docs/developer/package-integration/testing.md
index a0e13312f4c039173210082264b8132a55aa9bee..defec3d9f95d62ea8ffa6f6463dd050714b6bfe7 100644
--- a/docs/developer/package-integration/package-integration-testing.md
+++ b/docs/developer/package-integration/testing.md
@@ -11,7 +11,7 @@ Usually, Helm charts come with a set of Helm tests that can be run to test the d
## Prerequisites
-- Package helm chart with CI settings pointing to one of bigbang's [package pipelines](./package-integration-pipeline.md)
+- Package helm chart with CI settings pointing to one of bigbang's [package pipelines](./pipeline.md)
## Integration
diff --git a/docs/developer/package-integration/package-integration-upstream.md b/docs/developer/package-integration/upstream.md
similarity index 94%
rename from docs/developer/package-integration/package-integration-upstream.md
rename to docs/developer/package-integration/upstream.md
index c98d1fbd7aa82d0ac6f3b3851fc2b68c49faad3d..47f53fb92f1354e65fb4a3aafbf97ab7ca243e42 100644
--- a/docs/developer/package-integration/package-integration-upstream.md
+++ b/docs/developer/package-integration/upstream.md
@@ -162,8 +162,8 @@ To minimize maintenance, it is preferable to reuse existing Helm charts availabl
* <https://github.com/stefanprodan/podinfo>
## Learn More
- * [Application Overview](docs/overview.md)
- * [Other Documentation](docs/)
+ * [Application Overview\](docs/overview.md)
+ * [Other Documentation\](docs/)
## Pre-Requisites
@@ -185,17 +185,17 @@ To minimize maintenance, it is preferable to reuse existing Helm charts availabl
## Values
- | Key | Type | Default | Description |
- |-----|------|---------|-------------|
- | replicaCount | int | `1` | |
- | logLevel | string | `"info"` | |
- | host | string | `nil` | |
- | backend | string | `nil` | |
+ | Key | Type | Default | Description |
+ | ------------ | ------ | -------- | ----------- |
+ | replicaCount | int | `1` | |
+ | logLevel | string | `"info"` | |
+ | host | string | `nil` | |
+ | backend | string | `nil` | |
...
## Contributing
- Please see the [contributing guide](./CONTRIBUTING.md) if you are interested in contributing.
+ Please see the [contributing guide\](./CONTRIBUTING.md) if you are interested in contributing.
```
1. Commit changes
diff --git a/docs/encryption.md b/docs/encryption.md
index 2c35b700d2b2b57ed438641a40b8281acf1a4f55..64ca23d2802d5551395a32a905d950f961d44896 100644
--- a/docs/encryption.md
+++ b/docs/encryption.md
@@ -7,6 +7,7 @@ Big Bang follows a [GitOps](https://www.weave.works/technologies/gitops/) approa
## SOPS
[Secrets Operations (SOPS)](https://github.com/mozilla/sops) is used to securely encrypt values in YAML, JSON, ENV, INI and BINARY formats. Secrets, such as pull credentials or certificates, should be encrypted with SOPS prior to committing into a Git repository.
+
> The private key used in SOPS should **NEVER** be stored in Git along side the encrypted secrets.
SOPS supports the ability to [add multiple keys](https://dev.to/stack-labs/manage-your-secrets-in-git-with-sops-common-operations-118g) to the same file so multiple key pairs can use the same secret. This is useful for environments which may have different keys, but use the same secrets. For each key used, SOPS writes the public key, used to encrypt, and an encrypted copy of the data to the file. Decryption requires use of one of the private keys used. After editing, the embedded public keys are used to re-encrypt the file for all key pairs.
@@ -22,26 +23,27 @@ To setup Big Bang with SOPS, a key pair must be created. The private key is use
| [Google Cloud Platform (GCP) Key Management Service (KMS)](https://cloud.google.com/security-key-management) | [Link](https://github.com/mozilla/sops#encrypting-using-gcp-kms) |
| [HashiCorp Vault](https://www.vaultproject.io/) | [Link](https://github.com/mozilla/sops#23encrypting-using-azure-key-vault) |
-> *GPG is not recommended for production use because the private key can be misplaced or compromised too easily
+> \*GPG is not recommended for production use because the private key can be misplaced or compromised too easily
## Configure SOPS
SOPS uses `.sops.yaml` as a configuration file for which keys to use for newly created files. Once a file is created, the key fingerprints are stored in the file and must be re-keyed to use any changes to `.sops.yaml`.
1. Follow the [SOPS instructions](https://github.com/mozilla/sops#210using-sopsyaml-conf-to-select-kmspgp-for-new-files) to configure `.sops.yaml` based on the encryption method you used. Multiple keys of the same type can be added using the block scalar yaml construct, `>-`, and separating them by a comma and newline.
+
> If you are using the Big Bang sample files, make sure to remove the development Big Bang key.
2. Add the following regex to only encrypt data in the yaml files
- ```yaml
- creation_rules:
- - encrypted_regex: '^(data|stringData)$'
- ```
+ ```yaml
+ creation_rules:
+ - encrypted_regex: "^(data|stringData)$"
+ ```
3. Save `.sops.yaml` in the root of folder of your configuration
4. If you have existing secrets, use the following to re-key them with the configuration in `.sops.yaml`
- ```shell
+ ```shell
# You must have the old private key to rekey the file
sops updatekeys <encrypted file>
```
@@ -54,7 +56,7 @@ SOPS uses `.sops.yaml` as a configuration file for which keys to use for newly c
1. Deploy your SOPS private key to a secret named `sops-gpg` in the cluster
- ```shell
+ ```shell
gpg --export-secret-keys --armor <new key fingerprint> | kubectl create secret generic sops-gpg -n bigbang --from-file=yourkey.asc=/dev/stdin
```
@@ -62,22 +64,24 @@ SOPS uses `.sops.yaml` as a configuration file for which keys to use for newly c
1. Configure your KMS key(s) in your `.sops.yaml` by adding the target key's ARN to the `kms` field within each creation rule.
- ```yaml
- creation_rules:
- - encrypted_regex: '^(data|stringData)$'
- path_regex: ./dev/.*
- kms: '<kms_key_arn>'
- ```
+ ```yaml
+ creation_rules:
+ - encrypted_regex: "^(data|stringData)$"
+ path_regex: ./dev/.*
+ kms: "<kms_key_arn>"
+ ```
1. Ensure your cluster (specifically the `flux-system/flux-controller`) has access to the specified key.
+
1. For AWS deployments, this can be managed via IAM roles as described in the [SOPS documentation](https://github.com/mozilla/sops#28assuming-roles-and-using-kms-in-various-aws-accounts).
1. For non-AWS deployments
+
1. Create an AWS user with appropriate permissions as described in the [SOPS documentation](https://github.com/mozilla/sops#28assuming-roles-and-using-kms-in-various-aws-accounts).
1. Create a secret named `sops-aws-creds` in the cluster using the access creds from the target user:
- ```shell
- k create secret generic -n flux-system sops-aws-creds --from-literal=access_key_id=<key_id> --from-literal=access_key_secret=<key>
- ```
+ ```shell
+ k create secret generic -n flux-system sops-aws-creds --from-literal=access_key_id=<key_id> --from-literal=access_key_secret=<key>
+ ```
### GCP KMS
@@ -93,7 +97,7 @@ TBD - [This article](https://blog.doit-intl.com/injecting-secrets-from-aws-gcp-o
## Configure Big Bang
-Big Bang needs to know how to retrieve the private key so it can deploy the encrypted secrets from Git. Decryption configuration is placed in the top-level manifest (e.g. `dev.yaml`, `prod.yaml`) from the [Big Bang template](https://repo1.dso.mil/platform-one/big-bang/customers/template).
+Big Bang needs to know how to retrieve the private key so it can deploy the encrypted secrets from Git. Decryption configuration is placed in the top-level manifest (e.g. `dev.yaml`, `prod.yaml`) from the [Big Bang template](https://repo1.dso.mil/platform-one/big-bang/customers/template).
### GPG
@@ -134,31 +138,31 @@ Configure the flux-system `kustomize-controller` component with AWS credential e
```yaml
bases:
-- ../../base/flux
+ - ../../base/flux
patchesStrategicMerge:
-- |-
- apiVersion: apps/v1
- kind: Deployment
- metadata:
- name: kustomize-controller
- namespace: flux-system
- spec:
- template:
- spec:
- containers:
- - name: manager
- env:
- - name: AWS_ACCESS_KEY_ID
- valueFrom:
- secretKeyRef:
- name: sops-aws-creds
- key: access_key_id
- - name: AWS_SECRET_ACCESS_KEY
- valueFrom:
- secretKeyRef:
- name: sops-aws-creds
- key: access_key_secret
+ - |-
+ apiVersion: apps/v1
+ kind: Deployment
+ metadata:
+ name: kustomize-controller
+ namespace: flux-system
+ spec:
+ template:
+ spec:
+ containers:
+ - name: manager
+ env:
+ - name: AWS_ACCESS_KEY_ID
+ valueFrom:
+ secretKeyRef:
+ name: sops-aws-creds
+ key: access_key_id
+ - name: AWS_SECRET_ACCESS_KEY
+ valueFrom:
+ secretKeyRef:
+ name: sops-aws-creds
+ key: access_key_secret
```
> Values should come from the `sops-aws-creds` secret created in [AWS KMS](#aws-kms) above
diff --git a/docs/guides/deployment_scenarios/multiple_ingress.md b/docs/guides/deployment-scenarios/multiple-ingress.md
similarity index 100%
rename from docs/guides/deployment_scenarios/multiple_ingress.md
rename to docs/guides/deployment-scenarios/multiple-ingress.md
diff --git a/docs/guides/deployment_scenarios/quickstart.md b/docs/guides/deployment-scenarios/quickstart.md
similarity index 100%
rename from docs/guides/deployment_scenarios/quickstart.md
rename to docs/guides/deployment-scenarios/quickstart.md
diff --git a/docs/guides/deployment_scenarios/sso_quickstart.md b/docs/guides/deployment-scenarios/sso-quickstart.md
similarity index 99%
rename from docs/guides/deployment_scenarios/sso_quickstart.md
rename to docs/guides/deployment-scenarios/sso-quickstart.md
index fc49f44a5ac8a642c10fab15f43f38b14fd9dc73..719ffd7030ea722c45aac25a7b64531ed450fa13 100644
--- a/docs/guides/deployment_scenarios/sso_quickstart.md
+++ b/docs/guides/deployment-scenarios/sso-quickstart.md
@@ -621,7 +621,7 @@ twistlock:
EOF
helm upgrade --install bigbang \$HOME/bigbang/chart \
- --values \$HOME/bigbang/docs/example_configs/keycloak-dev-values.yaml \
+ --values \$HOME/bigbang/docs/assets/configs/example/keycloak-dev-values.yaml \
--values \$HOME/ib_creds.yaml \
--values \$HOME/keycloak_qs_demo_values.yaml \
--values https://repo1.dso.mil/platform-one/big-bang/bigbang/-/raw/master/chart/ingress-certs.yaml \
diff --git a/docs/guides/backups-and-migrations/nexus-migration-with-velero.md b/docs/guides/nexus-migration-with-velero.md
similarity index 98%
rename from docs/guides/backups-and-migrations/nexus-migration-with-velero.md
rename to docs/guides/nexus-migration-with-velero.md
index d6079ce2384cc62d48695e62ddd0c16d66616d13..5b8afc654b6204abbc69bf60c9acb8f1bbb5a6d9 100644
--- a/docs/guides/backups-and-migrations/nexus-migration-with-velero.md
+++ b/docs/guides/nexus-migration-with-velero.md
@@ -89,7 +89,7 @@ backups/nexus-ns-backup/velero-backup.json
```
Also ensure an EBS volume snapshot has been created and the Snapshot status is `Completed`.
-
+
## Restoring From Backup
@@ -131,7 +131,7 @@ Also ensure an EBS volume snapshot has been created and the Snapshot status is `
docker run -p 8000:8000 nexus-docker.bigbang.dev/repository/nexus-docker/doom-dos:latest
```
- 
+ 
## Appendix
diff --git a/docs/guides/prerequisites/README.md b/docs/guides/prerequisites/README.md
index 555c833e53ed1450adabe9fdaa6c01a29e18aa39..cd548b5254dac40cecc64f3e4b3fcba4337ab683 100644
--- a/docs/guides/prerequisites/README.md
+++ b/docs/guides/prerequisites/README.md
@@ -1,24 +1,25 @@
# Prerequisites:
-* How the Prerequisites docs are organized:
- * This README.md is meant to be a high level overview of prerequsites.
- * /docs/guides/prerequsites/(some_topic).md files are meant to offer more specific guidance on prerequisites while staying generic.
- * The /docs/guides/deployment_scenarios/(some_topic).md files may also offer additional details on prerequesites specific to the scenario.
-* Prerequisites vary depending on deployment scenario, thus a table is used to give an overview.
-* Note for future edits: The following table was generated using tablesgenerator.com/markdown_tables, recommended to copy the table's raw text contents, visit tablesgenerator.com/markdown_tables, and File -> Paste table data when edits are needed.
-| Prerequisites(rows) vs Deployment Scenarios(columns) | QuickStart | Internet Connected | Internet Disconnected |
-|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| **[Minimum Hardware Requirements](minimum_hardware_requirements.md)** (vCPU, Memory, and Disk Storage for Big Bang in a production system) | The quickstart's demo values are known to work against a single VM with 8 CPU cores, 32 GB ram, and a 100 GB root partition. (It should be possible to lower the helm values to fit on 4 CPU cores and 16 GB ram, and less disk space.) | Appropriately sized CSP instances (The hardware requirements vary greatly depending on the enabled packages, internal database use, replicas, nodes, and memory/cpu requests.) | Appropriately sized virtual or bare metal machines (The hardware requirements vary greatly depending on the enabled packages, internal database use, replicas, nodes, and memory/cpu requests.) |
-| **[OS Preconfigured](os_preconfiguration.md) and Prehardened** (OS and level of hardening required depends on AO) | Prerequisite Recommended: A non-hardened single VM, (the quickstart has copy pasteable commands to satisfy the configuration requirements.) | Prerequisite (CSPs usually have marketplaces with pre-hardened VM images) | Prerequisite (configured to AO's risk tolerance / mission needs) |
-| **[Kubernetes Distribution Preconfigured](kubernetes_preconfiguration.md) to Best Practices and Prehardened** (Any CNCF Kubernetes Distribution will work as long as an AO signs off on it) | k3d is recommended for demos (It can be quickly installed and setup using copy pasteable commands in the quickstart, ships with a dockerized LB, works on every cloud, and bare metal) | Prerequisite (https://repo1.dso.mil/platform-one/distros) | Prerequisite (users are responsible for airgap image import of container images needed by chosen Kubernetes Distribution) |
-| **Default Storage Class** ((for Dynamic PVCs), the SC needs to support RWX (Read Write Many) Access Mode to support HA deployment of all BigBang AddOns) | Presatisfied* (*if using k3d, which has dynamic local volume storage class baked in) | Prerequisite It's recommended that users start with a CSP specific or Kubernetes Distro provided storage class | Prerequisite [(These docs compare Cloud Agnostic Storage Solutions)](../../k8s-storage/README.md#kubernetes-storage-options) |
-| **Support for Automated Provisioning of Service Type Load Balancer** (is recommended) | Presatisfied* (*if using k3d, which ships with the ability to add flags to treat the VM's port 443 as Kubernetes Service of Type LB's port 443, automation in the quick start repo leverages these flags) | Prerequisite Kubernetes Distributions usually have CSPs specific flags you can pass to the kube-apiserver to support auto provisioning of CSP LBs. | Prerequisite [(See docs for guidance on bare metal and no IAM scenarios)](kubernetes_preconfiguration.md#service-of-type-load-balancer) |
-| **Access to Container Images** (IronBank Image Pull Credentials or AirGap import from .tar.gz's) | Prerequisite (Anyone can go to login.dso.mil, and self register against P1's SSO. That can be used to login to registry1.dso.mil to generate image pull credentials for the QuickStart) | BigBang customers are recommended to use ask their BB Customer Liaison's for an IronBank Image pull robot account, which lasts 6 months. | Prerequisite (Airgap import of container images, [BigBang Releases](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/releases) includes a .tar.gz of IronBank Images) |
-| **Customer Controlled Private Git Repo** (for GitOps, the Cluster needs network access & Credentials for Read Access to said Git Repo) | Substituted (the turn key demo uses a non GitOps workflow ) | Prerequisite (or follow Air gap docs) | Prerequisite (Air gap docs assist with provisioning an ssh based git repo) |
-| **Encrypt Secrets as Code** (Use SOPS + CSP KMS or PGP to encrypt secrets that need to be stored in the GitRepo) | Substituted (Quickstart demo leverages clear text secrets in local files / outside of git) | Prerequisite (CSP KMS and IAM is more secure that gpg key pair) | Prerequisite (Use CSP KMS if available, PGP works universally, [Flux requires the private PGP key to not have a passphrase](https://toolkit.fluxcd.io/guides/mozilla-sops/#generate-a-gpg-key)) |
-| **Install and Configure Flux** (Flux needs Git Repo Credentials & CSP IAM rights for KMS decryption or a kubernetes secret containing a private PGP decryption key) | Presatisfied (Quickstart demo only needs installation not configuration) | Prerequisite (see BigBang docs, [flux docs](https://toolkit.fluxcd.io/components/source/gitrepositories/#spec-examples) are also a good resource for this) | Prerequisite (see BigBang docs) |
-| **HTTPS Certificates** | Presatisfied (The quickstart deploys *.bigbang.dev HTTPS Certificate, from the bigbang repo, a non-expired cleartext copy of the cert is kept there for dev/demo purposes, alternatively mkcert can be used to generate demo certs for arbitrary DNS names that will only be trusted by the laptop that provisioned the mkcert cert) | Prerequisite (HTTPS cert is provided by consumer) | Prerequisite (HTTPS cert is provided by consumer) |
-| **DNS** | Edit your Laptop's host file (/etc/hosts, C:\Windows\System32\drivers\etc\hosts), or use something like AWS VPC Private DNS and [sshuttle](https://github.com/sshuttle/sshuttle) to point to host VM (if using k3d) | Prerequisite (point DNS names to Layer 4 CSP LB) | Prerequisite (point DNS names to L4 LB) |
-| **HTTPS Certificate, DNS Name, and hostnames in BigBang's helm values must match** (in order for Ingress to work correctly.) | QuickStart leverages `*.bigbang.dev` HTTPS cert, and the BigBang Helm Chart's values.yaml's hostname defaults to bigbang.dev, just need to ensure multiple hostfile entries like "grafana.bigbang.dev " exist, or if you have access to DNS a wildcard entry to map CNAME `*.bigbang.dev` to k3d VM's IP | Prerequisite (update bigbang helm values in git repo so hostnames match HTTPS cert) | Prerequisite (update bigbang helm values in git repo so hostnames match HTTPS cert) |
-| **SSO Identity Provider** (Prerequisite for SSO Authentication Proxy feature) | Presatisfied* (*depending on which quick start config is used), There's exists a demo SSO config that leverages P1's CAC enabled SSO, it's coded to only work for localhost to balance turn key demo functionality against security concerns. | Prerequisite (You don't have to use Keycloak, you can use any OIDC/SAML Identity Provider) | Prerequisite* (Install your own Keycloak cluster), leverage a pre-existing airgap SSO solution, or configure to not use SSO* if not needed for the use case) |
-| **Ops Team to integrate, configure, and maintain BigBang** (needed skillsets: DevOps IaC/CaC all the things, automate most the things, document the rest, linux administration, productionalization and maintenance of a Kubernetes Cluster.) | QuickStart Demo is designed to be self service. | Prerequisite (BigBang Customer Integration Engineers are available to help long term Ops teams.) | Prerequisite |
+- How the Prerequisites docs are organized:
+ - This README.md is meant to be a high level overview of prerequsites.
+ - /docs/guides/prerequsites/(some_topic).md files are meant to offer more specific guidance on prerequisites while staying generic.
+ - The /docs/guides/deployment_scenarios/(some_topic).md files may also offer additional details on prerequesites specific to the scenario.
+- Prerequisites vary depending on deployment scenario, thus a table is used to give an overview.
+- Note for future edits: The following table was generated using tablesgenerator.com/markdown_tables, recommended to copy the table's raw text contents, visit tablesgenerator.com/markdown_tables, and File -> Paste table data when edits are needed.
+
+| Prerequisites(rows) vs Deployment Scenarios(columns) | QuickStart | Internet Connected | Internet Disconnected |
+| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **[Minimum Hardware Requirements](minimum-hardware-requirements.md)** (vCPU, Memory, and Disk Storage for Big Bang in a production system) | The quickstart's demo values are known to work against a single VM with 8 CPU cores, 32 GB ram, and a 100 GB root partition. (It should be possible to lower the helm values to fit on 4 CPU cores and 16 GB ram, and less disk space.) | Appropriately sized CSP instances (The hardware requirements vary greatly depending on the enabled packages, internal database use, replicas, nodes, and memory/cpu requests.) | Appropriately sized virtual or bare metal machines (The hardware requirements vary greatly depending on the enabled packages, internal database use, replicas, nodes, and memory/cpu requests.) |
+| **[OS Preconfigured](os-preconfiguration.md) and Prehardened** (OS and level of hardening required depends on AO) | Prerequisite Recommended: A non-hardened single VM, (the quickstart has copy pasteable commands to satisfy the configuration requirements.) | Prerequisite (CSPs usually have marketplaces with pre-hardened VM images) | Prerequisite (configured to AO's risk tolerance / mission needs) |
+| **[Kubernetes Distribution Preconfigured](kubernetes-preconfiguration.md) to Best Practices and Prehardened** (Any CNCF Kubernetes Distribution will work as long as an AO signs off on it) | k3d is recommended for demos (It can be quickly installed and setup using copy pasteable commands in the quickstart, ships with a dockerized LB, works on every cloud, and bare metal) | Prerequisite (https://repo1.dso.mil/platform-one/distros) | Prerequisite (users are responsible for airgap image import of container images needed by chosen Kubernetes Distribution) |
+| **Default Storage Class** ((for Dynamic PVCs), the SC needs to support RWX (Read Write Many) Access Mode to support HA deployment of all BigBang AddOns) | Presatisfied* (*if using k3d, which has dynamic local volume storage class baked in) | Prerequisite It's recommended that users start with a CSP specific or Kubernetes Distro provided storage class | Prerequisite [(These docs compare Cloud Agnostic Storage Solutions)](../../k8s-storage.md#kubernetes-storage-options) |
+| **Support for Automated Provisioning of Service Type Load Balancer** (is recommended) | Presatisfied* (*if using k3d, which ships with the ability to add flags to treat the VM's port 443 as Kubernetes Service of Type LB's port 443, automation in the quick start repo leverages these flags) | Prerequisite Kubernetes Distributions usually have CSPs specific flags you can pass to the kube-apiserver to support auto provisioning of CSP LBs. | Prerequisite [(See docs for guidance on bare metal and no IAM scenarios)](kubernetes-preconfiguration.md#service-of-type-load-balancer) |
+| **Access to Container Images** (IronBank Image Pull Credentials or AirGap import from .tar.gz's) | Prerequisite (Anyone can go to login.dso.mil, and self register against P1's SSO. That can be used to login to registry1.dso.mil to generate image pull credentials for the QuickStart) | BigBang customers are recommended to use ask their BB Customer Liaison's for an IronBank Image pull robot account, which lasts 6 months. | Prerequisite (Airgap import of container images, [BigBang Releases](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/releases) includes a .tar.gz of IronBank Images) |
+| **Customer Controlled Private Git Repo** (for GitOps, the Cluster needs network access & Credentials for Read Access to said Git Repo) | Substituted (the turn key demo uses a non GitOps workflow ) | Prerequisite (or follow Air gap docs) | Prerequisite (Air gap docs assist with provisioning an ssh based git repo) |
+| **Encrypt Secrets as Code** (Use SOPS + CSP KMS or PGP to encrypt secrets that need to be stored in the GitRepo) | Substituted (Quickstart demo leverages clear text secrets in local files / outside of git) | Prerequisite (CSP KMS and IAM is more secure that gpg key pair) | Prerequisite (Use CSP KMS if available, PGP works universally, [Flux requires the private PGP key to not have a passphrase](https://toolkit.fluxcd.io/guides/mozilla-sops/#generate-a-gpg-key)) |
+| **Install and Configure Flux** (Flux needs Git Repo Credentials & CSP IAM rights for KMS decryption or a kubernetes secret containing a private PGP decryption key) | Presatisfied (Quickstart demo only needs installation not configuration) | Prerequisite (see BigBang docs, [flux docs](https://toolkit.fluxcd.io/components/source/gitrepositories/#spec-examples) are also a good resource for this) | Prerequisite (see BigBang docs) |
+| **HTTPS Certificates** | Presatisfied (The quickstart deploys \*.bigbang.dev HTTPS Certificate, from the bigbang repo, a non-expired cleartext copy of the cert is kept there for dev/demo purposes, alternatively mkcert can be used to generate demo certs for arbitrary DNS names that will only be trusted by the laptop that provisioned the mkcert cert) | Prerequisite (HTTPS cert is provided by consumer) | Prerequisite (HTTPS cert is provided by consumer) |
+| **DNS** | Edit your Laptop's host file (/etc/hosts, C:\Windows\System32\drivers\etc\hosts), or use something like AWS VPC Private DNS and [sshuttle](https://github.com/sshuttle/sshuttle) to point to host VM (if using k3d) | Prerequisite (point DNS names to Layer 4 CSP LB) | Prerequisite (point DNS names to L4 LB) |
+| **HTTPS Certificate, DNS Name, and hostnames in BigBang's helm values must match** (in order for Ingress to work correctly.) | QuickStart leverages `*.bigbang.dev` HTTPS cert, and the BigBang Helm Chart's values.yaml's hostname defaults to bigbang.dev, just need to ensure multiple hostfile entries like "grafana.bigbang.dev " exist, or if you have access to DNS a wildcard entry to map CNAME `*.bigbang.dev` to k3d VM's IP | Prerequisite (update bigbang helm values in git repo so hostnames match HTTPS cert) | Prerequisite (update bigbang helm values in git repo so hostnames match HTTPS cert) |
+| **SSO Identity Provider** (Prerequisite for SSO Authentication Proxy feature) | Presatisfied* (*depending on which quick start config is used), There's exists a demo SSO config that leverages P1's CAC enabled SSO, it's coded to only work for localhost to balance turn key demo functionality against security concerns. | Prerequisite (You don't have to use Keycloak, you can use any OIDC/SAML Identity Provider) | Prerequisite* (Install your own Keycloak cluster), leverage a pre-existing airgap SSO solution, or configure to not use SSO* if not needed for the use case) |
+| **Ops Team to integrate, configure, and maintain BigBang** (needed skillsets: DevOps IaC/CaC all the things, automate most the things, document the rest, linux administration, productionalization and maintenance of a Kubernetes Cluster.) | QuickStart Demo is designed to be self service. | Prerequisite (BigBang Customer Integration Engineers are available to help long term Ops teams.) | Prerequisite |
diff --git a/docs/guides/prerequisites/default_storageclass.md b/docs/guides/prerequisites/default-storageclass.md
similarity index 97%
rename from docs/guides/prerequisites/default_storageclass.md
rename to docs/guides/prerequisites/default-storageclass.md
index 6f3a8f0221c0ab9d9ddba6c5cee01d21fc736e08..ff9198983b11b118e178f6527f2b4fe30b1522b1 100644
--- a/docs/guides/prerequisites/default_storageclass.md
+++ b/docs/guides/prerequisites/default-storageclass.md
@@ -70,7 +70,7 @@ allowVolumeExpansion: true
## Bare Metal/Cloud Agnostic Store Class Notes
-* The BigBang Product team put together a [Comparison Matrix of a few Cloud Agnostic Storage Class offerings](../../k8s-storage/README.md#kubernetes-storage-options)
+* The BigBang Product team put together a [Comparison Matrix of a few Cloud Agnostic Storage Class offerings](../../k8s-storage.md#kubernetes-storage-options)
* Note: No storage class specific container images exist in IronBank at this time.
* Approved IronBank Images will show up in <https://registry1.dso.mil>
* <https://repo1.dso.mil/dsop> can be used to check status of IronBank images.
diff --git a/docs/guides/prerequisites/install_flux.md b/docs/guides/prerequisites/install-flux.md
similarity index 100%
rename from docs/guides/prerequisites/install_flux.md
rename to docs/guides/prerequisites/install-flux.md
diff --git a/docs/guides/prerequisites/kubernetes_preconfiguration.md b/docs/guides/prerequisites/kubernetes-preconfiguration.md
similarity index 97%
rename from docs/guides/prerequisites/kubernetes_preconfiguration.md
rename to docs/guides/prerequisites/kubernetes-preconfiguration.md
index 57b0061e88c9a3162f13f53af35aad4237860d13..28c204cc99866c8ef2039754621d2c5abe5089a7 100644
--- a/docs/guides/prerequisites/kubernetes_preconfiguration.md
+++ b/docs/guides/prerequisites/kubernetes-preconfiguration.md
@@ -8,7 +8,7 @@
* CoreDNS in the kube-system namespace should be HA with pod anti-affinity rules
* Master Nodes should be HA and tainted.
* Consider using a licensed Kubernetes Distribution with a support contract.
-* [A default storage class should exist](default_storageclass.md) to support dynamic provisioning of persistent volumes.
+* [A default storage class should exist](default-storageclass.md) to support dynamic provisioning of persistent volumes.
## Service of Type Load Balancer
@@ -111,7 +111,7 @@ Note: it was unsuccessfully attempted to apply this setting via modifications to
### RKE2
* RKE2 turns PSPs on by default (see above for tips on disabling)
-* RKE2 sets selinux to enforcing by default ([see os_preconfiguration.md for selinux config](os_preconfiguration.md))
+* RKE2 sets selinux to enforcing by default ([see os-preconfiguration.md for selinux config](os-preconfiguration.md))
Since BigBang makes several assumptions about volume and load balancing provisioning by default, it's vital that the rke2 cluster must be properly configured. The easiest way to do this is through the in tree cloud providers, which can be configured through the `rke2` configuration file such as:
diff --git a/docs/guides/prerequisites/minimum_hardware_requirements.md b/docs/guides/prerequisites/minimum-hardware-requirements.md
similarity index 91%
rename from docs/guides/prerequisites/minimum_hardware_requirements.md
rename to docs/guides/prerequisites/minimum-hardware-requirements.md
index 483da7ee2662d357e9f93f4ed87452c2e19538d0..022fe196f6eb3d205f53992b4861b98a7d793198 100644
--- a/docs/guides/prerequisites/minimum_hardware_requirements.md
+++ b/docs/guides/prerequisites/minimum-hardware-requirements.md
@@ -1,5 +1,5 @@
# Big Bang Minimum Hardware Requirements
-To calculate the minimum CPU, memory, and disk storage required to run Big Bang, open the [minimum hardware requirements Excel spreadsheet](./minimum_hardware_requirements.xlsx) and follow the instructions there. This will allow you to select exactly which packages and pods are enabled. In addition, it includes extra considerations to help you with sizing your cluster. The final values will be for the entire cluster and can be split between multiple nodes.
+To calculate the minimum CPU, memory, and disk storage required to run Big Bang, open the [minimum hardware requirements Excel spreadsheet](./minimum-hardware-requirements.xlsx) and follow the instructions there. This will allow you to select exactly which packages and pods are enabled. In addition, it includes extra considerations to help you with sizing your cluster. The final values will be for the entire cluster and can be split between multiple nodes.
When running across multiple availability zones, keep in mind that some of your nodes may be down for zone maintenance and the remaining nodes need to be able to handle the CPU, memory, and disk space for your cluster.
diff --git a/docs/guides/prerequisites/minimum_hardware_requirements.xlsx b/docs/guides/prerequisites/minimum-hardware-requirements.xlsx
similarity index 100%
rename from docs/guides/prerequisites/minimum_hardware_requirements.xlsx
rename to docs/guides/prerequisites/minimum-hardware-requirements.xlsx
diff --git a/docs/guides/prerequisites/os_preconfiguration.md b/docs/guides/prerequisites/os-preconfiguration.md
similarity index 100%
rename from docs/guides/prerequisites/os_preconfiguration.md
rename to docs/guides/prerequisites/os-preconfiguration.md
diff --git a/docs/guides/prerequisites/PodUsageInGrafana.md b/docs/guides/prerequisites/pod-usage-in-grafana.md
similarity index 82%
rename from docs/guides/prerequisites/PodUsageInGrafana.md
rename to docs/guides/prerequisites/pod-usage-in-grafana.md
index 966814016c400e8541a5056ac857aee10e27d6eb..f0d34669277733e2c5aab82ed0d01edd7b962dde 100644
--- a/docs/guides/prerequisites/PodUsageInGrafana.md
+++ b/docs/guides/prerequisites/pod-usage-in-grafana.md
@@ -17,7 +17,7 @@
Or [review password value within helm chart](https://repo1.dso.mil/platform-one/big-bang/apps/core/monitoring/-/blob/main/chart/values.yaml#L708)
1. Once logged in and directed to the home page, click the menu Dashboard and then select Manage. \
- 
+ 
1. From the Dashboard select Kubernetes/Compute Resource / Pod . \
This creates a dashboard to monitor the pod resource CPU Usage, CPU Throttling, CPU quota, Memory Usage, Memory Quota, etc. \
- 
+ 
diff --git a/docs/guides/using_bigbang/default_credentials.md b/docs/guides/using-bigbang/default-credentials.md
similarity index 100%
rename from docs/guides/using_bigbang/default_credentials.md
rename to docs/guides/using-bigbang/default-credentials.md
diff --git a/docs/guides/using_bigbang/image_pull_policy.md b/docs/guides/using-bigbang/image-pull-policy.md
similarity index 100%
rename from docs/guides/using_bigbang/image_pull_policy.md
rename to docs/guides/using-bigbang/image-pull-policy.md
diff --git a/docs/guides/using_bigbang/values_guide.md b/docs/guides/using-bigbang/values-guide.md
similarity index 100%
rename from docs/guides/using_bigbang/values_guide.md
rename to docs/guides/using-bigbang/values-guide.md
diff --git a/docs/k8s-storage.md b/docs/k8s-storage.md
new file mode 100644
index 0000000000000000000000000000000000000000..c54762addb1188e75ae9442ca1f4418622511bd2
--- /dev/null
+++ b/docs/k8s-storage.md
@@ -0,0 +1,122 @@
+# Kubernetes Storage Options
+
+Use this data to assist in your CSI decision. However, when using a cloud provider we suggest you use their Kubernetes CSI.
+
+## Feature Matrix
+
+| Product | BB Compatible | License Type | In Ironbank | RWX/RWM Support | Airgap Compatible | Cloud Agnostic |
+| --------------- | ------------- | ------------------------------------------------------------------------------------------------ | ----------- | ------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------- |
+| Amazon EBS CSI | **X** | Apache License 2.0 | | **X** | AWS Dependent | No |
+| Azure Disk CSI | Not Tested | Apache License 2.0 | | **X** | Azure Dependent | No |
+| Longhorn v1.1.0 | **X** | Apache License 2.0 | | **X** | **X** - [Docs](https://longhorn.io/docs/1.1.0/advanced-resources/deploy/airgap/) | Yes, uses host storage |
+| OpenEBS (jiva) | **X** | Apache License 2.0 | | **X** **[Alpha](https://docs.openebs.io/docs/next/rwm.html)** | Manual Work Required | Yes, uses host storage |
+| Rook-Ceph | **X** | Rook - Apache License 2.0. Ceph - dual licensed under the LGPL version 2.1 or 3.0 | | **X** | Manual Work Required | Yes, uses host storage |
+| Portworx | **X** | Tiered License - [See website](https://docs.portworx.com/reference/knowledge-base/px-licensing/) | | **X** | **X** - [Docs](https://docs.portworx.com/portworx-install-with-kubernetes/operate-and-maintain-on-kubernetes/pxcentral-onprem/install/px-central/) | Yes, uses host storage |
+
+## Benchmark Results
+
+Benchmarks were tested on AWS with GP2 ebs volumes using using FIO, see [example](./assets/configs/k8s-storage/benchmark.yaml)
+
+| Product | Random Read/Write IOPS | Average Latency (usec) | Sequential Read/Write | Mixed Random Read/Write IOPS |
+| --------------- | ------------------------------------- | ---------------------- | --------------------- | ---------------------------- |
+| Amazon EBS CSI | 2997/2996. BW: 128MiB/s / 128MiB/s | 1331.61 | 129MiB/s / 131MiB/s | 7203/2390 |
+| Azure Disk CSI | | | |
+| Longhorn v1.1.0 | 6155/1551 BW: 230MiB/s / 96.3MiB/s | 1042.53 | 319MiB/s / 130MiB/s | 3804/1267 |
+| OpenEBS (jiva) | 2183/770. BW: 76.8MiB/s / 45.8MiB/s | 2059.55 | 132MiB/s / 98.2MiB/s | 1590/533 |
+| Rook-Ceph | 10.7k/3205. BW: 503MiB/s / 148MiB/s | 548.36/s | 496MiB/s / 154MiB/s | 6664/2228 |
+| Portworx 2.6 | 3016/19.3k. BW: 74.5MiB/s / 85.1MiB/s | 1337.31 | 113MiB/s / 124MiB/s | 35.1k/11.1k |
+
+## Amazon EBS CSI
+
+[Website/Docs](https://docs.aws.amazon.com/eks/latest/userguide/ebs-csi.html)
+
+### Requirements
+
+- Must be using AWS
+
+### Notes
+
+- Apache License 2.0
+- Very easy to install and use, apply CSI spec and you are ready.
+
+## Azure Disk CSI
+
+[Website/Docs](https://docs.microsoft.com/en-us/azure/aks/azure-disk-csi)
+
+### Requirements
+
+- Must be using Azure
+
+### Notes
+
+- Apache License 2.0
+- Very easy to install and use, apply CSI spec and you are ready.
+
+## Longhorn
+
+[Website/Docs](https://longhorn.io/)
+
+### Requirements
+
+- RWX requires `nfs-common` to be installed on the nodes. [Longhorn RWX Docs](https://longhorn.io/docs/1.1.0/advanced-resources/rwx-workloads/)
+
+### Notes
+
+- Apache License 2.0
+- Easiest to install
+- Built-in backup tool.
+- Documented airgap install process. [Docs](https://longhorn.io/docs/1.1.0/advanced-resources/deploy/airgap/)
+- GUI provides data and observability; replica status, cluster health status, backup status, and backup initiation/recovery.
+- Native backup to S3 or NFS
+
+## OpenEBS
+
+[Website/Docs](https://openebs.io/)
+
+### Requirements
+
+- Blank and un-partitioned attached disk(s)
+- RWX is in Alpha and requires work. [OpenEBS RWX Docs](https://docs.openebs.io/docs/next/rwm.html)
+
+### Notes
+
+- Very flexible, supports multiple storage designs.
+
+| Application requirements | Storage Type | OpenEBS Volumes |
+| --------------------------------------------------------------------------------------------- | ------------------------------------ | ------------------------------ |
+| Low Latency, High Availability, Synchronous replication, Snapshots, Clones, Thin provisioning | SSDs/Cloud Volumes | OpenEBS Mayastor |
+| High Availability, Synchronous replication, Snapshots, Clones, Thin provisioning | Disks/SSDs/Cloud Volumes | OpenEBS cStor |
+| High Availability, Synchronous replication, Thin provisioning | hostpath or external mounted storage | OpenEBS Jiva |
+| Low latency, Local PV | hostpath or external mounted storage | Dynamic Local PV - Hostpath |
+| Low latency, Local PV | Disks/SSDs/Cloud Volumes | Dynamic Local PV - Device |
+| Low latency, Local PV, Snapshots, Clones | Disks/SSDs/Cloud Volumes | OpenEBS Dynamic Local PV - ZFS |
+
+## Rook-Ceph
+
+[Website/Docs](https://rook.io/)
+
+### Requirements
+
+- Blank and un-partitioned attached disk(s)
+
+### Notes
+
+- Rook - Apache License 2.0.
+- Ceph - dual licensed under the LGPL version 2.1 or 3.0
+- Very Fast
+
+## Portworx
+
+[Website/Docs](https://docs.portworx.com/portworx-install-with-kubernetes/)
+
+### Requirements
+
+- Blank and un-partitioned attached disk(s)
+
+### Notes
+
+- Portworx Essentials is free **up to** 5nodes, 5TB Storage, 500 volumes
+- Portworx Enterprise and PX-Backup require paid licenses
+- Best Mixed IOPS, average read/write performance
+- Install is very picky about the container runtime hostpath
+- Tested on Konvoy 1.6.1 due to Portworx issues when using RKE2
diff --git a/docs/k8s-storage/README.md b/docs/k8s-storage/README.md
deleted file mode 100644
index 957ca83b9caed2c171aba03338d6e5f4796de805..0000000000000000000000000000000000000000
--- a/docs/k8s-storage/README.md
+++ /dev/null
@@ -1,122 +0,0 @@
-# Kubernetes Storage Options
-
-Use this data to assist in your CSI decision. However, when using a cloud provider we suggest you use their Kubernetes CSI.
-
-## Feature Matrix
-
-| Product | BB Compatible | License Type | In Ironbank | RWX/RWM Support | Airgap Compatible | Cloud Agnostic |
-| --------- | --------- | --------- | --------- | --------- | --------- | --------- |
-Amazon EBS CSI | **X** | Apache License 2.0 | | **X** | AWS Dependent | No |
-Azure Disk CSI | Not Tested | Apache License 2.0 | | **X** | Azure Dependent | No |
-Longhorn v1.1.0 | **X** | Apache License 2.0 | | **X** | **X** - [Docs](https://longhorn.io/docs/1.1.0/advanced-resources/deploy/airgap/) | Yes, uses host storage |
-OpenEBS (jiva) | **X** | Apache License 2.0 | | **X** **[Alpha](https://docs.openebs.io/docs/next/rwm.html)** | Manual Work Required | Yes, uses host storage |
-Rook-Ceph | **X** | Rook - Apache License 2.0. Ceph - dual licensed under the LGPL version 2.1 or 3.0 | | **X** | Manual Work Required | Yes, uses host storage |
-Portworx | **X** | Tiered License - [See website](https://docs.portworx.com/reference/knowledge-base/px-licensing/) | | **X** | **X** - [Docs](https://docs.portworx.com/portworx-install-with-kubernetes/operate-and-maintain-on-kubernetes/pxcentral-onprem/install/px-central/) | Yes, uses host storage |
-
-## Benchmark Results
-
-Benchmarks were tested on AWS with GP2 ebs volumes using using FIO, see [example](./benchmark.yaml)
-
-| Product | Random Read/Write IOPS | Average Latency (usec) | Sequential Read/Write | Mixed Random Read/Write IOPS |
-| --------- | --------- | --------- | --------- | --------- |
-Amazon EBS CSI | 2997/2996. BW: 128MiB/s / 128MiB/s | 1331.61 | 129MiB/s / 131MiB/s | 7203/2390
-Azure Disk CSI | | | |
-Longhorn v1.1.0 | 6155/1551 BW: 230MiB/s / 96.3MiB/s | 1042.53 | 319MiB/s / 130MiB/s | 3804/1267
-OpenEBS (jiva) | 2183/770. BW: 76.8MiB/s / 45.8MiB/s | 2059.55 | 132MiB/s / 98.2MiB/s | 1590/533
-Rook-Ceph | 10.7k/3205. BW: 503MiB/s / 148MiB/s | 548.36/s | 496MiB/s / 154MiB/s | 6664/2228
-Portworx 2.6 | 3016/19.3k. BW: 74.5MiB/s / 85.1MiB/s | 1337.31 | 113MiB/s / 124MiB/s | 35.1k/11.1k
-
-## Amazon EBS CSI
-
-[Website/Docs](https://docs.aws.amazon.com/eks/latest/userguide/ebs-csi.html)
-
-### Requirements
-
-- Must be using AWS
-
-### Notes
-
-- Apache License 2.0
-- Very easy to install and use, apply CSI spec and you are ready.
-
-## Azure Disk CSI
-
-[Website/Docs](https://docs.microsoft.com/en-us/azure/aks/azure-disk-csi)
-
-### Requirements
-
-- Must be using Azure
-
-### Notes
-
-- Apache License 2.0
-- Very easy to install and use, apply CSI spec and you are ready.
-
-## Longhorn
-
-[Website/Docs](https://longhorn.io/)
-
-### Requirements
-
-- RWX requires `nfs-common` to be installed on the nodes. [Longhorn RWX Docs](https://longhorn.io/docs/1.1.0/advanced-resources/rwx-workloads/)
-
-### Notes
-
-- Apache License 2.0
-- Easiest to install
-- Built-in backup tool.
-- Documented airgap install process. [Docs](https://longhorn.io/docs/1.1.0/advanced-resources/deploy/airgap/)
-- GUI provides data and observability; replica status, cluster health status, backup status, and backup initiation/recovery.
-- Native backup to S3 or NFS
-
-## OpenEBS
-
-[Website/Docs](https://openebs.io/)
-
-### Requirements
-
-- Blank and un-partitioned attached disk(s)
-- RWX is in Alpha and requires work. [OpenEBS RWX Docs](https://docs.openebs.io/docs/next/rwm.html)
-
-### Notes
-
-- Very flexible, supports multiple storage designs.
-
-Application requirements | Storage Type | OpenEBS Volumes
-| --------- | --------- | --------- |
-Low Latency, High Availability, Synchronous replication, Snapshots, Clones, Thin provisioning | SSDs/Cloud Volumes | OpenEBS Mayastor
-High Availability, Synchronous replication, Snapshots, Clones, Thin provisioning | Disks/SSDs/Cloud Volumes | OpenEBS cStor
-High Availability, Synchronous replication, Thin provisioning | hostpath or external mounted storage | OpenEBS Jiva
-Low latency, Local PV | hostpath or external mounted storage | Dynamic Local PV - Hostpath
-Low latency, Local PV | Disks/SSDs/Cloud Volumes | Dynamic Local PV - Device
-Low latency, Local PV, Snapshots, Clones | Disks/SSDs/Cloud Volumes | OpenEBS Dynamic Local PV - ZFS
-
-## Rook-Ceph
-
-[Website/Docs](https://rook.io/)
-
-### Requirements
-
-- Blank and un-partitioned attached disk(s)
-
-### Notes
-
-- Rook - Apache License 2.0.
-- Ceph - dual licensed under the LGPL version 2.1 or 3.0
-- Very Fast
-
-## Portworx
-
-[Website/Docs](https://docs.portworx.com/portworx-install-with-kubernetes/)
-
-### Requirements
-
-- Blank and un-partitioned attached disk(s)
-
-### Notes
-
-- Portworx Essentials is free **up to** 5nodes, 5TB Storage, 500 volumes
-- Portworx Enterprise and PX-Backup require paid licenses
-- Best Mixed IOPS, average read/write performance
-- Install is very picky about the container runtime hostpath
-- Tested on Konvoy 1.6.1 due to Portworx issues when using RKE2
diff --git a/docs/production.md b/docs/production.md
index 9c572784b4717e49cc2cae1f72955eec2a872eb2..9cea12ce951f5f5abad48a83c4537f28f5ef7210 100644
--- a/docs/production.md
+++ b/docs/production.md
@@ -3,6 +3,7 @@
[[_TOC_]]
## Gatekeeper
+
The gatekeeper `values` section should resemble below when deploying to production.
```yaml
@@ -21,11 +22,11 @@ gatekeeper:
# -- Values to passthrough to the gatekeeper chart: https://repo1.dso.mil/platform-one/big-bang/apps/core/policy.git
values:
- violations:
- allowedDockerRegistries:
- match:
- excludedNamespaces:
- - kube-system # ignored as the kubernetes distro cannot be controlled
+ violations:
+ allowedDockerRegistries:
+ match:
+ excludedNamespaces:
+ - kube-system # ignored as the kubernetes distro cannot be controlled
# -- Post Renderers. See docs/postrenders.md
postRenderers: []
@@ -61,14 +62,17 @@ spec:
```
## Gitlab
+
This section provides suggested settings for Gitlab operational/production environments.
### Use external database service
-For production deployments you must externalize the database service. BigBang will pass through the most common value overrides to the Gitlab Package chart.
+
+For production deployments you must externalize the database service. BigBang will pass through the most common value overrides to the Gitlab Package chart.
Disable the internal postgres by configuring an external database in the BigBang values.
+
```yaml
addons:
- gitlab:
+ gitlab:
database:
host:
port:
@@ -77,9 +81,11 @@ addons:
password:
```
-### Use external object storage
+### Use external object storage
+
For production deployments you must externalize object storage service. BigBang will pass through the most common value overrides to the Gitlab Package chart.
Disable the internal MinIO instance by configuring an external object storage service.
+
```yaml
addons:
gitlab:
@@ -94,7 +100,9 @@ addons:
```
### Flux settings
+
Large Gitlab installations should increase the Gitlab specific HelmRelease timeout value to around 30m to 45m and the Gitlab specific HelmRelease retries value should be adjusted to around 8 to 10.
+
```yaml
addons:
gitlab:
@@ -106,8 +114,10 @@ addons:
```
### Kubernetes resource request/limit settings
+
K8s resource requests/limits for webservice and gitaly workloads should be increased from the defaults. Gitlab engineers state predicting Gitaly's resource consumption is very difficult, and will require testing to find the applicable limits/requests for each individual installation. See this [Gitlab Epic](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more information. See the [gitlab/docs/k8s-resources.md](https://repo1.dso.mil/platform-one/big-bang/apps/developer-tools/gitlab/-/blob/main/docs/k8s-resources.md) for a list of all possible configuration values. Use BigBang values overrides to change the Gitlab resource settings.
Recommended starting point:
+
```yaml
addons:
gitlab:
@@ -132,45 +142,57 @@ addons:
```
### Backup and rename gitlab-rails-secret
-An operational deployment of Gitlab should backup and re-create the Gitlab Rails Encryption information as a secret with a different name as [documented here](https://docs.gitlab.com/charts/installation/secrets.html#gitlab-rails-secret). Using a custom secret name can help prevent accidental overwriting.
+
+An operational deployment of Gitlab should backup and re-create the Gitlab Rails Encryption information as a secret with a different name as [documented here](https://docs.gitlab.com/charts/installation/secrets.html#gitlab-rails-secret). Using a custom secret name can help prevent accidental overwriting.
To make the secret creation easier, the existing secret can be copied and modified with a different name.
+
```bash
kubectl get secret/gitlab-rails-secret -n gitlab -o yaml > gitlab-rails-custom-secret.yaml
```
+
Edit the file and change the name of the secret. Example:
+
```yaml
apiVersion: v1
kind: Secret
metadata:
name: gitlab-rails-custom-secret
```
-Use GitOps configuration as code (CaC) and commit the custom rails secret to your GitOps repository. You should encrypt the custom rails secret keys in the GitOps repository to preserve security.
+
+Use GitOps configuration as code (CaC) and commit the custom rails secret to your GitOps repository. You should encrypt the custom rails secret keys in the GitOps repository to preserve security.
Then the following Gitlab helm chart value `global.railsSecrets.secret` can be overridden to point to the custom rails secret.
+
```yaml
addons:
gitlab:
values:
global:
railsSecrets:
- secret: gitlab-rails-custom-secret
+ secret: gitlab-rails-custom-secret
```
+
The custom rails secret should be backed up somewhere secure outside the cluster if not included in your GitOps code repository.
**If the Kubernetes gitlab-rails-secret happens to get overwritten Gitlab will no longer be able to access the encrypted data in the database.**
You will get errors like this in the logs.
+
```text
OpenSSL::Cipher::CipherError ()
```
-Many things break when this happens and the recovery is ugly with serious user impacts.
+
+Many things break when this happens and the recovery is ugly with serious user impacts.
At a minimum an operational deployment of Gitlab should export and save the gitlab-rails-secret somewhere secure outside the cluster.
+
```bash
kubectl get secret/gitlab-rails-secret -n gitlab -o yaml > cya.yaml
```
## Vault
+
This section provides suggested settings for Vault operational/production environments. Vault is a large complicated application and has many options that cannot adequately be covered here. Vault has significant security risks if not properly configured and administrated. Please consult the upstream [Vault documentation](https://learn.hashicorp.com/tutorials/vault/kubernetes-raft-deployment-guide?in=vault/kubernetes#configure-vault-helm-chart) as the ultimate authority. The following is an example operational/production config using a passthrough istio ingress gateway, high availability, auto-unseal, and raft for distributed filesystem persistence. Consult the BigBang Vault Package helm repo [/docs/production-ha.md](https://repo1.dso.mil/platform-one/big-bang/apps/sandbox/vault/-/blob/main/docs/production-ha.md) for more information.
+
```yaml
istio:
enabled: true
@@ -184,7 +206,7 @@ istio:
passthrough:
ingressGateway: "passthrough-ingressgateway"
hosts:
- - "*.{{ .Values.domain }}"
+ - "*.{{ .Values.domain }}"
tls:
mode: "PASSTHROUGH"
@@ -251,7 +273,7 @@ addons:
enabled: true
setNodeId: true
- # these values should be encrypted to prevent the kms_key_id from being revealed
+ # these values should be encrypted to prevent the kms_key_id from being revealed
config: |
ui = true
@@ -275,14 +297,14 @@ addons:
leader_client_key_file = "/vault/tls/tls.key"
leader_tls_servername = "vault.bigbang.dev"
}
-
+
retry_join {
leader_api_addr = "https://vault-vault-1.vault-vault-internal:8200"
leader_client_cert_file = "/vault/tls/tls.crt"
leader_client_key_file = "/vault/tls/tls.key"
leader_tls_servername = "vault.bigbang.dev"
}
-
+
retry_join {
leader_api_addr = "https://vault-vault-2.vault-vault-internal:8200"
leader_client_cert_file = "/vault/tls/tls.crt"
@@ -303,4 +325,4 @@ addons:
}
service_registration "kubernetes" {}
-```
\ No newline at end of file
+```
diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md
index 7d222cd41e772a5459c023447889de818068a52b..2d0de146e14336d415a6c57650c36611cb1b4706 100644
--- a/docs/troubleshooting.md
+++ b/docs/troubleshooting.md
@@ -1,13 +1,4 @@
-# Appendix B - Big Bang Troubleshooting
-
-- [Appendix B - Big Bang Troubleshooting](#appendix-b---big-bang-troubleshooting)
- - [Iron Bank authentication](#iron-bank-authentication)
- - [Flux install](#flux-install)
- - [Git Repository](#git-repository)
- - [ConfigMap or Secrets](#configmap-or-secrets)
- - [Helm Release](#helm-release)
- - [Kustomization](#kustomization)
- - [Packages](#packages)
+# Big Bang Troubleshooting
Big Bang can take a long time to run. After making changes, it could take 10-15 minutes to take effect. Use the [sync.sh](../scripts/sync.sh) script to speed this up.
diff --git a/docs/understanding_bigbang/README.md b/docs/understanding-bigbang/README.md
similarity index 100%
rename from docs/understanding_bigbang/README.md
rename to docs/understanding-bigbang/README.md
diff --git a/docs/understanding_bigbang/kube-apiserver_webhooks_diagram.md b/docs/understanding-bigbang/kube-apiserver-webhooks-diagram.md
similarity index 95%
rename from docs/understanding_bigbang/kube-apiserver_webhooks_diagram.md
rename to docs/understanding-bigbang/kube-apiserver-webhooks-diagram.md
index 2168dfe71dba92c9cd57332d25939a485ca1caf2..0021c36d443c6636ace6a1c57d6eab69f736eb15 100644
--- a/docs/understanding_bigbang/kube-apiserver_webhooks_diagram.md
+++ b/docs/understanding-bigbang/kube-apiserver-webhooks-diagram.md
@@ -15,7 +15,7 @@
## BigBang Kubernetes API Server Webhooks Diagram
-
+
### Notes
diff --git a/docs/understanding_bigbang/licensing_expectations.md b/docs/understanding-bigbang/licensing-expectations.md
similarity index 100%
rename from docs/understanding_bigbang/licensing_expectations.md
rename to docs/understanding-bigbang/licensing-expectations.md
diff --git a/docs/understanding_bigbang/logs_data_flow_diagram.md b/docs/understanding-bigbang/logs-data-flow-diagram.md
similarity index 90%
rename from docs/understanding_bigbang/logs_data_flow_diagram.md
rename to docs/understanding-bigbang/logs-data-flow-diagram.md
index 280a5d891fa57ae468f28418f960a9038288ad2a..6cebbce51d071ac66531b244dcdf15b4d395a07d 100644
--- a/docs/understanding_bigbang/logs_data_flow_diagram.md
+++ b/docs/understanding-bigbang/logs-data-flow-diagram.md
@@ -4,7 +4,7 @@
## Kubernetes Pod Logs Data Flow Diagram
-
+
| Line Number | Protocol | Port | Description |
| --- | --- | --- | --- |
diff --git a/docs/understanding_bigbang/metrics_data_flow_diagram.md b/docs/understanding-bigbang/metrics-data-flow-diagram.md
similarity index 85%
rename from docs/understanding_bigbang/metrics_data_flow_diagram.md
rename to docs/understanding-bigbang/metrics-data-flow-diagram.md
index ebe992bd7998a1e41532b02b3248eda7f8f19305..019ae2dd8f7e80f8ed2c219c23f0564505335610 100644
--- a/docs/understanding_bigbang/metrics_data_flow_diagram.md
+++ b/docs/understanding-bigbang/metrics-data-flow-diagram.md
@@ -4,7 +4,7 @@
## Prometheus Metrics Data Flow Diagram
-
+
| Line Number | Protocol | Port | Description |
| --- | --- | --- | --- |
diff --git a/docs/understanding_bigbang/network_encryption_and_ingress_diagram.md b/docs/understanding-bigbang/network-encryption-and-ingress-diagram.md
similarity index 98%
rename from docs/understanding_bigbang/network_encryption_and_ingress_diagram.md
rename to docs/understanding-bigbang/network-encryption-and-ingress-diagram.md
index 8e30212647dcdfce6c4836ef8a6c53a656a18301..0fb642e15bfe5184e1f074bd10e6fba99d7db790 100644
--- a/docs/understanding_bigbang/network_encryption_and_ingress_diagram.md
+++ b/docs/understanding-bigbang/network-encryption-and-ingress-diagram.md
@@ -8,7 +8,7 @@
## Big Bang Network Ingress Diagram
-
+
### Notes