diff --git a/docs/developer/develop-package.md b/docs/developer/develop-package.md
index 2c5a4ffa46e8bb0f29ad4ee775adc28406bde494..6c242baaa90283c71cb8af3274c1af9cd930e4b1 100644
--- a/docs/developer/develop-package.md
+++ b/docs/developer/develop-package.md
@@ -7,56 +7,56 @@ Package is the term we use for an application that has been prepared to be deplo
1. Create a "main" branch that will serve as the master branch.
1. There are two ways to start a new Package.
- A. If there is no upstream helm chart we create a helm chart from scratch. Here is a T3 video that demonstrates creating a new helm chart. Create a directory called "chart" in your repo, change to the chart directory, and scaffold a new chart in the chart directory
+ 1. If there is no upstream helm chart we create a helm chart from scratch. Here is a T3 video that demonstrates creating a new helm chart. Create a directory called "chart" in your repo, change to the chart directory, and scaffold a new chart in the chart directory
- ```shell
- # Scaffold new helm chart
- mkdir chart
- cd chart
- helm create name-of-your-application
- ```
+ ```shell
+ # Scaffold new helm chart
+ mkdir chart
+ cd chart
+ helm create name-of-your-application
+ ```
- B. If there is an existing upstream chart we will use it and modify it. Essentially we create a "fork" of the upstream code. Use kpt to import the helm chart code into your repository. Note that kpt is not used to keep the Package code in sync with the upstream chart. It is a one time pull just to document where the upstream chart code came from. Kpt will generate a Kptfile that has the details. Do not manually create the "chart" directory. The kpt command will create it. Here is an example from when Gitlab Package was created. It is a good idea to push a commit "initial upstream chart with no changes" so you can refer back to the original code while you are developing.
+ 2. If there is an existing upstream chart we will use it and modify it. Essentially we create a "fork" of the upstream code. Use kpt to import the helm chart code into your repository. Note that kpt is not used to keep the Package code in sync with the upstream chart. It is a one time pull just to document where the upstream chart code came from. Kpt will generate a Kptfile that has the details. Do not manually create the "chart" directory. The kpt command will create it. Here is an example from when Gitlab Package was created. It is a good idea to push a commit "initial upstream chart with no changes" so you can refer back to the original code while you are developing.
- ```shell
- kpt pkg get https://gitlab.com/gitlab-org/charts/gitlab.git@v4.8.0 chart
- ```
+ ```shell
+ kpt pkg get https://gitlab.com/gitlab-org/charts/gitlab.git@v4.8.0 chart
+ ```
1. Run a helm dependency update that will download any external sub-chart dependencies. Commit any *.tgz files that are downloaded into the "charts" directory. The reason for doing this is that BigBang Packages must be able to be installed in an air-gap without any internet connectivity.
- ```shell
- helm dependency update
- ```
+ ```shell
+ helm dependency update
+ ```
1. Edit the Chart.yaml and set the chart ```version:``` number to be compliant with the charter versioning which is {UpstreamChartVersion}-bb.{BigBangVersion}. Note that the chart version is not the same thing as the application version. If this is a patch to an existing Package chart then increment the {BigBangVersion}. Here is an example from Gitlab Runner.
- ```yaml
- apiVersion: v1
- name: gitlab-runner
- version: 0.19.2-bb.3
- appVersion: 13.2.2
- description: GitLab Runner
- ```
+ ```yaml
+ apiVersion: v1
+ name: gitlab-runner
+ version: 0.19.2-bb.3
+ appVersion: 13.2.2
+ description: GitLab Runner
+ ```
1. In the values.yaml replace public upstream images with IronBank hardened images. The image version should be compatible with the chart version. Here is a command to identify the images that need to be changed.
- ```shell
- # list images
- helm template <releasename> ./chart -n <namespace> -f chart/values.yaml | grep image:
- ```
+ ```shell
+ # list images
+ helm template <releasename> ./chart -n <namespace> -f chart/values.yaml | grep image:
+ ```
- Add the "imagePullSecrets" tag if not already there. You can still test without the private-registry secret existing if your k8s cluster is configured with the pull credentials. Here is an example from Gitlab Package.
+ Add the "imagePullSecrets" tag if not already there. You can still test without the private-registry secret existing if your k8s cluster is configured with the pull credentials. Here is an example from Gitlab Package.
- ```yaml
- registry:
- enabled: true
- host: "registry.bigbang.dev"
- image:
- repository: registry1.dso.mil/ironbank/gitlab/gitlab/gitlab-container-registry
- tag: 13.7.2
- pullSecrets:
- - name: private-registry
- ```
+ ```yaml
+ registry:
+ enabled: true
+ host: "registry.bigbang.dev"
+ image:
+ repository: registry1.dso.mil/ironbank/gitlab/gitlab/gitlab-container-registry
+ tag: 13.7.2
+ pullSecrets:
+ - name: private-registry
+ ```
1. Add a VirtualService if your application has a back-end API or a front-end GUI. Create the VirtualService in the sub-directory "chart/templates/bigbang/VirtualService.yaml". You will need to manually create the "bigbang" directory. It is convenient to copy VirtualService code from one of the other Packages and then modify it. You should be able to load the application in your browser if all the configuration is correct.
@@ -68,43 +68,43 @@ Package is the term we use for an application that has been prepared to be deplo
1. Add the following markdown files to complete the Package. Reference other that Packages for examples of how to create them.
- ```shell
- CHANGELOG.md < standard history of changes made
- CODEOWNERS < list of the code maintainers. Minimum of two people from separate organizations
- CONTRIBUTING.md < instructions for how to contribute to the project
- README.md < introduction and high level information
- ```
+ ```shell
+ CHANGELOG.md < standard history of changes made
+ CODEOWNERS < list of the code maintainers. Minimum of two people from separate organizations
+ CONTRIBUTING.md < instructions for how to contribute to the project
+ README.md < introduction and high level information
+ ```
1. Create a top-level tests directory and inside put a test-values.yaml file that includes any special values overrides that are needed for CI pipeline testing. Refer to other packages for examples. But this is specific to what is needed for your package.
- ```shell
- mkdir tests
- touch test-values.yaml
- ```
+ ```shell
+ mkdir tests
+ touch test-values.yaml
+ ```
1. At a high level, a Package structure should look like this when you are finished
- ```plaintext
- ├── chart/
- └── templates/
- └── bigbang/
- ├── networkpolicies/
- ├── egress-*.yaml
- └── ingress-*.yaml
- └── virtualservice.yaml
- ├── tests/
- ├── cypress/
- └── scripts/
- ├── docs/
- ├── documentation-file-1.md
- └── documentation-file-2.md
- ├── tests/
- └── test-values.yaml
- ├── CHANGELOG.md
- ├── CODEOWNERS
- ├── CONTRIBUTING.md
- └── README.md
- ```
+ ```plaintext
+ ├── chart/
+ └── templates/
+ └── bigbang/
+ ├── networkpolicies/
+ ├── egress-*.yaml
+ └── ingress-*.yaml
+ └── virtualservice.yaml
+ ├── tests/
+ ├── cypress/
+ └── scripts/
+ ├── docs/
+ ├── documentation-file-1.md
+ └── documentation-file-2.md
+ ├── tests/
+ └── test-values.yaml
+ ├── CHANGELOG.md
+ ├── CODEOWNERS
+ ├── CONTRIBUTING.md
+ └── README.md
+ ```
1. Merging code should require approval from a minimum of 2 codeowners. To setup merge requests to work properly with CODEOWNERS approval change these settings in your project:
Under Settings → General → Merge Request Approvals, change "Any eligible user" "Approvals required" to 1. Also ensure that "Require new approvals when new commits are added to an MR" is checked.
@@ -112,63 +112,63 @@ Under Settings → Repository → Protected Branches, add the main branch with "
Under Settings → Repository → Default Branch, ensure that main is selected.
1. Development Testing Cycle: Test your Package chart by deploying with helm. Test frequently so you don't pile up multiple layers of errors. The goal is for Packages to be deployable independently of the bigbang chart. Most upstream helm charts come with internal services like a database that can be toggled on or off. If available use them for testing and CI pipelines. In some cases this is not an option. You can manually deploy required in-cluster services in order to complete your development testing.
- Here is an example of an in-cluster postgres database
-
- ```shell
- helm repo add bitnami https://charts.bitnami.com/bitnami
- helm install postgres bitnami/postgresql -n postgres --create-namespace --set postgresqlPostgresPassword=postgres --set postgresqlPassword=postgres
- # test it
- kubectl run postgresql-postgresql-client --rm --tty -i --restart='Never' --namespace default --image bitnami/postgresql --env="PGPASSWORD=postgres" --command -- psql --host postgres-postgresql-headless.postgres.svc.cluster.local -U postgres -d postgres -p 5432
- # Postgres commands
+ Here is an example of an in-cluster postgres database
+
+ ```shell
+ helm repo add bitnami https://charts.bitnami.com/bitnami
+ helm install postgres bitnami/postgresql -n postgres --create-namespace --set postgresqlPostgresPassword=postgres --set postgresqlPassword=postgres
+ # test it
+ kubectl run postgresql-postgresql-client --rm --tty -i --restart='Never' --namespace default --image bitnami/postgresql --env="PGPASSWORD=postgres" --command -- psql --host postgres-postgresql-headless.postgres.svc.cluster.local -U postgres -d postgres -p 5432
+ # Postgres commands
\l < list tables
\du < list users
\q < quit
- ```
-
- Here is an example of an in-cluster object storage service using MinIO (api compatible with AWS S3 storage)
-
- ```shell
- helm repo add minio https://helm.min.io/
- helm install minio minio/minio --set accessKey=myaccesskey --set secretKey=mysecretkey -n minio --create-namespace
- # test and configure it
- kubectl run minio-mc-client --rm --tty -i --restart='Never' --namespace default --image minio/mc --command -- bash
- # MinIo commands
- mc alias set minio http://minio.minio.svc.cluster.local:9000 myaccesskey mysecretkey < set a connection alias
- mc mb minio/myBucket < make a bucket
- mc ls minio < list the buckets
- ```
-
- Create a local directory on your workstation where you store your helm values override files. Don't make test changes in the Package values.yaml because they could accidentally be committed. The most convenient location is in a sibling directory next to the Package repo. Here is an example directory structure:
-
- ```plaintext
- ├── PackageRepo/
- └── overrides/
- └── override-values.yaml
- ```
-
- Here are the dev test steps you can iterate:
-
- ```shell
- # Test that the helm chart templates successfully and examine the output to insure expected results
- helm template <releasename> ./chart -n <namespace> -f ../overrides/override-values.yaml
- # Deploy with helm
- helm upgrade -i <releasename> ./chart -n <namespace> --create-namespace -f ../overrides/override-values.yaml
- # Conduct testing
- # Tear down
- helm delete <releasename> -n <namespace>
- # Manually delete the namespace to insure that everything is gone
- kubectl delete ns <namespace>
- ```
+ ```
+
+ Here is an example of an in-cluster object storage service using MinIO (api compatible with AWS S3 storage)
+
+ ```shell
+ helm repo add minio https://helm.min.io/
+ helm install minio minio/minio --set accessKey=myaccesskey --set secretKey=mysecretkey -n minio --create-namespace
+ # test and configure it
+ kubectl run minio-mc-client --rm --tty -i --restart='Never' --namespace default --image minio/mc --command -- bash
+ # MinIo commands
+ mc alias set minio http://minio.minio.svc.cluster.local:9000 myaccesskey mysecretkey < set a connection alias
+ mc mb minio/myBucket < make a bucket
+ mc ls minio < list the buckets
+ ```
+
+ Create a local directory on your workstation where you store your helm values override files. Don't make test changes in the Package values.yaml because they could accidentally be committed. The most convenient location is in a sibling directory next to the Package repo. Here is an example directory structure:
+
+ ```plaintext
+ ├── PackageRepo/
+ └── overrides/
+ └── override-values.yaml
+ ```
+
+ Here are the dev test steps you can iterate:
+
+ ```shell
+ # Test that the helm chart templates successfully and examine the output to insure expected results
+ helm template <releasename> ./chart -n <namespace> -f ../overrides/override-values.yaml
+ # Deploy with helm
+ helm upgrade -i <releasename> ./chart -n <namespace> --create-namespace -f ../overrides/override-values.yaml
+ # Conduct testing
+ # Tear down
+ helm delete <releasename> -n <namespace>
+ # Manually delete the namespace to insure that everything is gone
+ kubectl delete ns <namespace>
+ ```
1. Wait to create a git tag release until integration testing with BigBang chart is completed. You will very likely discover more Package changes that are needed during BigBang integration. When you are confident that the Package code is complete, squash commits and rebase your development branch with the "main" branch.
- ```shell
- git rebase origin/main
- git reset $(git merge-base origin/main $(git rev-parse --abbrev-ref HEAD))
- git add -A
- git commit -m "feat: example conventional commit"
- git push --force
- ```
+ ```shell
+ git rebase origin/main
+ git reset $(git merge-base origin/main $(git rev-parse --abbrev-ref HEAD))
+ git add -A
+ git commit -m "feat: example conventional commit"
+ git push --force
+ ```
1. Then, create a merge request to branch "main"
@@ -181,24 +181,24 @@ In some instances you may wish to manually create a private-registry secret in t
1. The first way is to add the secret manually using kubectl. This method is useful for standalone package testing/development.
- ```shell
- kubectl create secret docker-registry private-registry --docker-server="https://registry1.dso.mil" --docker-username='Username' --docker-password="CLI secret" --docker-email=<your-email> --namespace=<package-namespace>
- ```
+ ```shell
+ kubectl create secret docker-registry private-registry --docker-server="https://registry1.dso.mil" --docker-username='Username' --docker-password="CLI secret" --docker-email=<your-email> --namespace=<package-namespace>
+ ```
2. The second is to create a yaml file containing the secret and apply it during a helm install. This method is applicable when installing your new package as part of the Big Bang chart. In this example the file name is "reg-creds.yaml":
-Create the file with the secret contents:
+ Create the file with the secret contents:
- ```yaml
- registryCredentials:
- registry: registry1.dso.mil
- username: ""
- password: ""
- email: ""
- ```
+ ```yaml
+ registryCredentials:
+ registry: registry1.dso.mil
+ username: ""
+ password: ""
+ email: ""
+ ```
-Then include a reference to your file during your helm install command by adding the below `-f` to your Big Bang install command:
+ Then include a reference to your file during your helm install command by adding the below `-f` to your Big Bang install command:
- ```shell
- -f reg-creds.yaml
- ```
+ ```shell
+ -f reg-creds.yaml
+ ```
diff --git a/docs/developer/package-integration/flux.md b/docs/developer/package-integration/flux.md
index 5594e687608a6330889e9d717aaf847344fad333..2af8e765f2be3a0ba1cb10729c5e8c277a3961d3 100644
--- a/docs/developer/package-integration/flux.md
+++ b/docs/developer/package-integration/flux.md
@@ -272,47 +272,47 @@ To validate that the Helm chart is working, perform the following steps to deplo
1. Disable all packages that are enabled by default in Big Bang by adding the following to `bigbang/values.yaml`
- ```yaml
- # Network Policies
- networkPolicies:
- enabled: false
-
- # Istio
- istioOperator:
- enabled: false
- istio:
- enabled: false
-
- # Gatekeeper
- gatekeeper:
- enabled: false
- clusterAuditor:
- enabled: false
-
- # Logging
- eckOperator:
- enabled: false
- elasticsearchKibana:
- enabled: false
- fluentbit:
- enabled: false
-
- # Monitoring
- monitoring:
- enabled: false
-
- # Other Tools
- jaeger:
- enabled: false
- kiali:
- enabled: false
- twistlock:
- enabled: false
- ```
+ ```yaml
+ # Network Policies
+ networkPolicies:
+ enabled: false
+
+ # Istio
+ istioOperator:
+ enabled: false
+ istio:
+ enabled: false
+
+ # Gatekeeper
+ gatekeeper:
+ enabled: false
+ clusterAuditor:
+ enabled: false
+
+ # Logging
+ eckOperator:
+ enabled: false
+ elasticsearchKibana:
+ enabled: false
+ fluentbit:
+ enabled: false
+
+ # Monitoring
+ monitoring:
+ enabled: false
+
+ # Other Tools
+ jaeger:
+ enabled: false
+ kiali:
+ enabled: false
+ twistlock:
+ enabled: false
+ ```
1. To enable Big Bang packages that are disabled by default, add the appropriate code block for the Big Bang package from the Big Bang helm chart [values file](https://repo1.dso.mil/big-bang/bigbang/-/blob/master/chart/values.yaml).
- - For example, if you want to test a package's functionality with MinIO, you would add this block from the Big Bang helm chart values file to the package repo's `bigbang/values.yaml` file
+ - For example, if you want to test a package's functionality with MinIO, you would add this block from the Big Bang helm chart values file to the package repo's `bigbang/values.yaml` file
```yaml
addons:
@@ -322,75 +322,75 @@ To validate that the Helm chart is working, perform the following steps to deplo
minio:
enabled: true
```
-
- - This would deploy MinIO in the `integration` stage with default configurations
-
- - Any values that are present in the Big Bang helm chart values file can be configured here also. The pipeline merges these two files into a single values file before deployment, so the package repo's `bigbang/values.yaml` provides a way to configure a Third Party or Sandbox package along with Big Bang package configurations in a single values file.
- - A `tests/test-values.yaml` in a package repo can be used as override values for the pipeline. It allows pipeline-specific configurations so that the package's `chart/values.yaml` doesn't have to be changed.
+ - This would deploy MinIO in the `integration` stage with default configurations
+
+ - Any values that are present in the Big Bang helm chart values file can be configured here also. The pipeline merges these two files into a single values file before deployment, so the package repo's `bigbang/values.yaml` provides a way to configure a Third Party or Sandbox package along with Big Bang package configurations in a single values file.
+
+ - A `tests/test-values.yaml` in a package repo can be used as override values for the pipeline. It allows pipeline-specific configurations so that the package's `chart/values.yaml` doesn't have to be changed.
1. Install flux using the [instructions from Big Bang](https://repo1.dso.mil/big-bang/bigbang/-/blob/master/docs/guides/deployment-scenarios/quickstart.md#step-8-install-flux).
1. Install the package using the bigbang Helm chart
- ```shell
- helm upgrade -i -n bigbang --create-namespace -f ~/bigbang/chart/values.yaml -f bigbang/values.yaml bigbang-podinfo bigbang
- ```
+ ```shell
+ helm upgrade -i -n bigbang --create-namespace -f ~/bigbang/chart/values.yaml -f bigbang/values.yaml bigbang-podinfo bigbang
+ ```
1. Watch the `GitRepository`, `HelmRelease`, and `Pods`:
- ```shell
- watch kubectl get gitrepo,hr,po -A
- ```
+ ```shell
+ watch kubectl get gitrepo,hr,po -A
+ ```
1. Troubleshoot any errors
- ```shell
- kubectl get events -A
- ```
+ ```shell
+ kubectl get events -A
+ ```
- > If you are using a private Git repository or pulling images from a private image repository, you will need to add credentials into the `git.credentials.username`/`git.credentials.password` and/or `registryCredentials.username`/`registryCredentials.password` using the `--set` option for Helm.
+ > If you are using a private Git repository or pulling images from a private image repository, you will need to add credentials into the `git.credentials.username`/`git.credentials.password` and/or `registryCredentials.username`/`registryCredentials.password` using the `--set` option for Helm.
1. Cleanup cluster
- ```shell
- helm delete -n bigbang bigbang-podinfo
- ```
+ ```shell
+ helm delete -n bigbang bigbang-podinfo
+ ```
1. Add the following to `bigbang/README.md` to document this Helm charts usage:
- ```markdown
- # Big Bang compatible Helm chart
+ ```markdown
+ # Big Bang compatible Helm chart
- This helm chart deploys the application using the same methods and values as Big Bang.
+ This helm chart deploys the application using the same methods and values as Big Bang.
- ## Prerequisites
+ ## Prerequisites
- - Kubernetes cluster matching [Big Bang's Prerequisites](https://repo1.dso.mil/big-bang/bigbang/-/tree/master/docs/guides/prerequisites)
- - [FluxCD](https://fluxcd.io/) running in the cluster
- - The [Big Bang git repository](https://repo1.dso.mil/big-bang/bigbang) cloned into `~/bigbang`
- - [Helm](https://helm.sh/docs/intro/install/)
+ - Kubernetes cluster matching [Big Bang's Prerequisites](https://repo1.dso.mil/big-bang/bigbang/-/tree/master/docs/guides/prerequisites)
+ - [FluxCD](https://fluxcd.io/) running in the cluster
+ - The [Big Bang git repository](https://repo1.dso.mil/big-bang/bigbang) cloned into `~/bigbang`
+ - [Helm](https://helm.sh/docs/intro/install/)
- ## Usage
+ ## Usage
- ### Installation
+ ### Installation
- 1. Install Big Bang
- `helm upgrade -i -n bigbang --create-namespace -f ~/bigbang/chart/values.yaml -f bigbang/values.yaml bigbang ~/bigbang/chart`
- 1. Install this chart
- `helm upgrade -i -n bigbang --create-namespace -f ~/bigbang/chart/values.yaml -f bigbang/values.yaml bigbang-podinfo bigbang`
+ 1. Install Big Bang
+ `helm upgrade -i -n bigbang --create-namespace -f ~/bigbang/chart/values.yaml -f bigbang/values.yaml bigbang ~/bigbang/chart`
+ 1. Install this chart
+ `helm upgrade -i -n bigbang --create-namespace -f ~/bigbang/chart/values.yaml -f bigbang/values.yaml bigbang-podinfo bigbang`
- ### Removal
+ ### Removal
- `helm delete -n bigbang bigbang-podinfo`
+ `helm delete -n bigbang bigbang-podinfo`
- ```
+ ```
1. Commit your changes
- > If you are developing something different than `podinfo`, run `grep -ir podinfo` to make sure your replaced all of the instances with your application name.
+ > If you are developing something different than `podinfo`, run `grep -ir podinfo` to make sure your replaced all of the instances with your application name.
- ```shell
- git add -A
- git commit -m "feat: added bigbang helm chart"
- git push
- ```
+ ```shell
+ git add -A
+ git commit -m "feat: added bigbang helm chart"
+ git push
+ ```
diff --git a/docs/developer/package-integration/helm-standards.md b/docs/developer/package-integration/helm-standards.md
index e4219e4326c18ddc64791decda558a1999245cac..0f81c445e2a4fa7d0419740318a0a0a14623de35 100644
--- a/docs/developer/package-integration/helm-standards.md
+++ b/docs/developer/package-integration/helm-standards.md
@@ -42,16 +42,16 @@ For another example in using the [`kube-prometheus-stack`](https://github.com/pr
* The BigBang chart should prevent the use of a database bundled as part of the package chart by default, and warn if an end user uses one anyways.
* There should be a database section under the package configuration that matches the following section
- ```yaml
- database:
- # Entering connection info will enable external database and will auto-create any required secrets.
- host: ""
- port: ""
- username: ""
- password: ""
- database: ""
- type: "" # Optional. One of mysql, mssql, postgres, mongo if ther
- ```
+ ```yaml
+ database:
+ # Entering connection info will enable external database and will auto-create any required secrets.
+ host: ""
+ port: ""
+ username: ""
+ password: ""
+ database: ""
+ type: "" # Optional. One of mysql, mssql, postgres, mongo if ther
+ ```
* Monitoring
* Charts should expect a value `monitoring.enabled` to be set by the BigBang chart to conditionally create monitoring components (`ServiceMonitors`, `PodMonitors`, etc). This value should default to false
diff --git a/docs/developer/package-integration/monitoring.md b/docs/developer/package-integration/monitoring.md
index 8bb03e085180f4e11373d4345fb173f3accd877d..9b712d58bad677485bf273b884a6338af075b558 100644
--- a/docs/developer/package-integration/monitoring.md
+++ b/docs/developer/package-integration/monitoring.md
@@ -108,66 +108,66 @@ Dashboards are important for administrators to understand what is happening in y
1. Create a dashboard
- Some packages or maintainers provide Grafana dashboards upstream, otherwise you can search [Grafana's Dashboard Repository](https://grafana.com/grafana/dashboards/) for a relevant Dashboard. If there is already a ready-made Grafana dashboard for your package provided upstream, you should use [Kpt](https://googlecontainertools.github.io/kpt/installation/) to sync it into monitoring package (for example flux provides the JSON dashboards in their upstream repo):
+ Some packages or maintainers provide Grafana dashboards upstream, otherwise you can search [Grafana's Dashboard Repository](https://grafana.com/grafana/dashboards/) for a relevant Dashboard. If there is already a ready-made Grafana dashboard for your package provided upstream, you should use [Kpt](https://googlecontainertools.github.io/kpt/installation/) to sync it into monitoring package (for example flux provides the JSON dashboards in their upstream repo):
- ```shell
- # There isn't a dashboard for podinfo, so we use flux as an example here
- kpt pkg get https://github.com/fluxcd/flux2.git//manifests/monitoring/grafana/dashboards@v0.9.1 chart/dashboards/
- ```
+ ```shell
+ # There isn't a dashboard for podinfo, so we use flux as an example here
+ kpt pkg get https://github.com/fluxcd/flux2.git//manifests/monitoring/grafana/dashboards@v0.9.1 chart/dashboards/
+ ```
- If you need to create your own dashboard, open Grafana and use `Create > Dashboard`. Add a panel and setup the query to pull custom data from your package or general data about your pods (e.g. container_processes). After you have saved your dashboard in Grafana, use `Share (icon) > Export` to save the dashboard to a .json file in `chart/dashboards`. You can leave the `Export for sharing externally` slider off.
+ If you need to create your own dashboard, open Grafana and use `Create > Dashboard`. Add a panel and setup the query to pull custom data from your package or general data about your pods (e.g. container_processes). After you have saved your dashboard in Grafana, use `Share (icon) > Export` to save the dashboard to a .json file in `chart/dashboards`. You can leave the `Export for sharing externally` slider off.
1. We will store dashboards in a ConfigMap for Grafana's sidecar to parse. Create a ConfigMapList in `chart/templates/bigbang/dashboards.yaml` to store all of the dashboards:
- ```yaml
- {{- $pkg := "podinfo" }}
- {{- $files := .Files.Glob "dashboards/*.json" }}
- {{- if and .Values.serviceMonitor.enabled $files }}
- apiVersion: v1
- kind: ConfigMapList
- items:
- {{- range $path, $fileContents := $files }}
- {{- $dashboardName := regexReplaceAll "(^.*/)(.*)\\.json$" $path "${2}" }}
- - apiVersion: v1
- kind: ConfigMap
- metadata:
- name: {{ printf "%s-%s" $pkg $dashboardName | trunc 63 | trimSuffix "-" }}
- namespace: {{ default $.Release.Namespace $.Values.serviceMonitor.dashboards.namespace }}
- labels:
- {{- if $.Values.serviceMonitor.dashboards.label }}
- {{ $.Values.serviceMonitor.dashboards.label }}: "1"
- {{- end }}
- app: {{ $pkg }}-grafana
- {{- include (printf "%s.labels" $pkg) $ | nindent 6 }}
- data:
- {{ $dashboardName }}.json: {{ $.Files.Get $path | toJson }}
- {{- end }}
- {{- end }}
- ```
-
- > Podinfo's Helm chart already had a key for monitoring named `serviceMonitor`. You may need to use a different key or create one named `monitoring`.
+ ```yaml
+ {{- $pkg := "podinfo" }}
+ {{- $files := .Files.Glob "dashboards/*.json" }}
+ {{- if and .Values.serviceMonitor.enabled $files }}
+ apiVersion: v1
+ kind: ConfigMapList
+ items:
+ {{- range $path, $fileContents := $files }}
+ {{- $dashboardName := regexReplaceAll "(^.*/)(.*)\\.json$" $path "${2}" }}
+ - apiVersion: v1
+ kind: ConfigMap
+ metadata:
+ name: {{ printf "%s-%s" $pkg $dashboardName | trunc 63 | trimSuffix "-" }}
+ namespace: {{ default $.Release.Namespace $.Values.serviceMonitor.dashboards.namespace }}
+ labels:
+ {{- if $.Values.serviceMonitor.dashboards.label }}
+ {{ $.Values.serviceMonitor.dashboards.label }}: "1"
+ {{- end }}
+ app: {{ $pkg }}-grafana
+ {{- include (printf "%s.labels" $pkg) $ | nindent 6 }}
+ data:
+ {{ $dashboardName }}.json: {{ $.Files.Get $path | toJson }}
+ {{- end }}
+ {{- end }}
+ ```
+
+ > Podinfo's Helm chart already had a key for monitoring named `serviceMonitor`. You may need to use a different key or create one named `monitoring`.
1. Commit your dashboard files:
- ```shell
- git add -A
- git commit -m "feat: Grafana dashboards"
- git push
- ```
+ ```shell
+ git add -A
+ git commit -m "feat: Grafana dashboards"
+ git push
+ ```
1. If your package is being integrated as a supported application in BigBang, you can add your Dashboards to the core monitoring package.
- Create a new folder within `chart/dashboards/APP_NAME` and sync your JSON files for your dashboard(s) there, whether using KPT from a Github repo or individual files from Grafana's Dashboard Repository.
+ Create a new folder within `chart/dashboards/APP_NAME` and sync your JSON files for your dashboard(s) there, whether using KPT from a Github repo or individual files from Grafana's Dashboard Repository.
- Commit your dashboard files:
+ Commit your dashboard files:
- ```shell
- git add -A
- git commit -m "feat: Adding APP_NAME Grafana Dashboards"
- git push
- ```
+ ```shell
+ git add -A
+ git commit -m "feat: Adding APP_NAME Grafana Dashboards"
+ git push
+ ```
- Any JSON dashboards in the `chart/dashboards` folder automatically get created and imported into the monitoring stack via the Operator.
+ Any JSON dashboards in the `chart/dashboards` folder automatically get created and imported into the monitoring stack via the Operator.
## Validation
diff --git a/docs/developer/package-integration/pipeline.md b/docs/developer/package-integration/pipeline.md
index d3b930e7bb3ec6cb3c6b38860f25cced7c6014aa..98776241ededd1d4cf87f04f1870d76922e7ffc1 100644
--- a/docs/developer/package-integration/pipeline.md
+++ b/docs/developer/package-integration/pipeline.md
@@ -24,73 +24,72 @@ Pipelines provide rapid feedback to changes in our Helm chart as we develop and
- [sandbox](https://repo1.dso.mil/platform-one/big-bang/pipeline-templates/pipeline-templates/-/blob/master/pipelines/sandbox.yaml)
- [third-party](https://repo1.dso.mil/platform-one/big-bang/pipeline-templates/pipeline-templates/-/blob/master/pipelines/third-party.yaml)
-1. The pipeline **requires** that all images are stored in either Iron Bank (`registry1.dso.mil`) or Repo1 (`registry.dso.mil`). In some cases, you may be able to substitute images already in Iron Bank for the ones in the Helm chart. For example, images for `curl`, `kubectl` or `jq` can use `registry1.dso.mil/ironbank/big-bang/base`. If you have not already submitted your containers to Iron Bank, [start the process](https://repo1.dso.mil/dsop/dccscr/-/blob/master/README.md). While you are working your way to Iron Bank approval, you can temporarily put the images in `registry.dso.mil` for development by doing the following:
- > Check if the Container Registry is on by navigating to `https://repo1.dso.mil/platform-one/big-bang/apps/sandbox/<your project>/container_registry`. If you get a 404 error, you need to request a Maintainer turn this feature on in your project via Settings > General > Visibility > Container Registry.
+The pipeline **requires** that all images are stored in either Iron Bank (`registry1.dso.mil`) or Repo1 (`registry.dso.mil`). In some cases, you may be able to substitute images already in Iron Bank for the ones in the Helm chart. For example, images for `curl`, `kubectl` or `jq` can use `registry1.dso.mil/ironbank/big-bang/base`. If you have not already submitted your containers to Iron Bank, [start the process](https://repo1.dso.mil/dsop/dccscr/-/blob/master/README.md). While you are working your way to Iron Bank approval, you can temporarily put the images in `registry.dso.mil` for development by doing the following:
- ```shell
- # Image Info
- export IMGSRC_REPO=docker.io
- export IMGSRC_PROJ=stefanprodan
- export IMGDST_REPO=registry.dso.mil
- export IMGDST_PROJ=platform-one/big-bang/apps/sandbox/podinfo
- export IMGNAME=podinfo
- export IMGTAG=6.0.0
+> Check if the Container Registry is on by navigating to `https://repo1.dso.mil/platform-one/big-bang/apps/sandbox/<your project>/container_registry`. If you get a 404 error, you need to request a Maintainer turn this feature on in your project via Settings > General > Visibility > Container Registry.
- # Pull image locally
- docker pull $IMGSRC_REPO/$IMGSRC_PROJ/$IMGNAME:$IMGTAG
+```shell
+# Image Info
+export IMGSRC_REPO=docker.io
+export IMGSRC_PROJ=stefanprodan
+export IMGDST_REPO=registry.dso.mil
+export IMGDST_PROJ=platform-one/big-bang/apps/sandbox/podinfo
+export IMGNAME=podinfo
+export IMGTAG=6.0.0
- # Retag image
- docker tag $IMGSRC_REPO/$IMGSRC_PROJ/$IMGNAME:$IMGTAG $IMGDST_REPO/$IMGDST_PROJ/$IMGNAME:$IMGTAG
+# Pull image locally
+docker pull $IMGSRC_REPO/$IMGSRC_PROJ/$IMGNAME:$IMGTAG
- # Login in docker registry
- docker login $IMGDST_REPO
+# Retag image
+docker tag $IMGSRC_REPO/$IMGSRC_PROJ/$IMGNAME:$IMGTAG $IMGDST_REPO/$IMGDST_PROJ/$IMGNAME:$IMGTAG
- # Push to registry
- docker push $IMGDST_REPO/$IMGDST_PROJ/$IMGNAME:$IMGTAG
- ```
+# Login in docker registry
+docker login $IMGDST_REPO
-1. Update `chart/values.yaml` with either the `registry1.dso.mil` or `registry.dso.mil` for images. For example:
+# Push to registry
+docker push $IMGDST_REPO/$IMGDST_PROJ/$IMGNAME:$IMGTAG
+```
- ```yaml
- image:
- repository: registry.dso.mil/platform-one/big-bang/apps/sandbox/podinfo/podinfo
- tag: 6.0.0
- ```
+1. Update `chart/values.yaml` with either the `registry1.dso.mil` or `registry.dso.mil` for images. For example:
+ ```yaml
+ image:
+ repository: registry.dso.mil/platform-one/big-bang/apps/sandbox/podinfo/podinfo
+ tag: 6.0.0
+ ```
1. Update the repo's CI/CD settings to call the pipeline (`Settings > CI/CD > General pipelines > Expand > CI/CD configuration file`).
-
For Bigbang
- ```plaintext
- pipelines/bigbang-package.yaml@platform-one/big-bang/pipeline-templates/pipeline-templates:master
- ```
+ ```plaintext
+ pipelines/bigbang-package.yaml@platform-one/big-bang/pipeline-templates/pipeline-templates:master
+ ```
For Third party
- ```plaintext
- pipelines/third-party.yaml@platform-one/big-bang/pipeline-templates/pipeline-templates:master
- ```
+ ```plaintext
+ pipelines/third-party.yaml@platform-one/big-bang/pipeline-templates/pipeline-templates:master
+ ```
For Sandbox
- ```plaintext
- pipelines/sandbox.yaml@platform-one/big-bang/pipeline-templates/pipeline-templates:master
- ```
+ ```plaintext
+ pipelines/sandbox.yaml@platform-one/big-bang/pipeline-templates/pipeline-templates:master
+ ```
1. Add overlay values for testing into `tests/test-values.yaml`. This will be where you add values needed for running in the pipeline. For now it can be a blank, placeholder.
1. Commit the changes
- ```shell
- git add -A
- git commit -m "feat: package pipeline"
- git push
- ```
+ ```shell
+ git add -A
+ git commit -m "feat: package pipeline"
+ git push
+ ```
1. Big Bang requires a Merge Request to run the pipeline. Open a MR to merge your branch into the main branch.
- > You will need to add `SKIP UPDATE CHECK` and `SKIP UPGRADE` into the title of the first MR or it will fail. Until you have a baseline Helm chart and CHANGELOG in place, these stages need to be skipped.
+ > You will need to add `SKIP UPDATE CHECK` and `SKIP UPGRADE` into the title of the first MR or it will fail. Until you have a baseline Helm chart and CHANGELOG in place, these stages need to be skipped.
1. The pipeline will install the package, run any Helm tests (`chart/tests`), and run any custom tests (`tests`).
diff --git a/docs/developer/package-integration/service-mesh.md b/docs/developer/package-integration/service-mesh.md
index 471daee4edb1b44ea2ff8d5b3f54b33a804e2b87..dd797943e4e99e3235271db4b5ffaed461080fdb 100644
--- a/docs/developer/package-integration/service-mesh.md
+++ b/docs/developer/package-integration/service-mesh.md
@@ -193,31 +193,31 @@ Test the following items to ensure Istio is working properly with your applicati
1. Verify syntax and resolve errors:
- ```shell
- helm template -n bigbang -f ~/bigbang/chart/values.yaml -f bigbang/values.yaml bigbang-podinfo bigbang
- ```
+ ```shell
+ helm template -n bigbang -f ~/bigbang/chart/values.yaml -f bigbang/values.yaml bigbang-podinfo bigbang
+ ```
1. Commit changes
- ```shell
- git add -A
- git commit -m "feat: Istio integration"
- git push
- ```
+ ```shell
+ git add -A
+ git commit -m "feat: Istio integration"
+ git push
+ ```
1. Install Big Bang core with `bigbang.dev` certificate
- ```shell
- # Iron Bank credentials are required since you are pulling Big Bang images from Iron Bank
- helm upgrade -i -n bigbang --create-namespace -f ~/bigbang/chart/values.yaml -f ~/bigbang/chart/ingress-certs.yaml -f bigbang/values.yaml --set registryCredentials.username=<your Iron Bank username> --set registryCredentials.password=<your Iron Bank PAT> bigbang ~/bigbang/chart
- ```
+ ```shell
+ # Iron Bank credentials are required since you are pulling Big Bang images from Iron Bank
+ helm upgrade -i -n bigbang --create-namespace -f ~/bigbang/chart/values.yaml -f ~/bigbang/chart/ingress-certs.yaml -f bigbang/values.yaml --set registryCredentials.username=<your Iron Bank username> --set registryCredentials.password=<your Iron Bank PAT> bigbang ~/bigbang/chart
+ ```
1. Install the package
- ```shell
- # Iron Bank credentials are optional until we migrate the package to an Iron Bank image
- helm upgrade -i -n bigbang --create-namespace -f ~/bigbang/chart/values.yaml -f ~/bigbang/chart/ingress-certs.yaml -f bigbang/values.yaml --set registryCredentials.username=<your Iron Bank username> --set registryCredentials.password=<your Iron Bank PAT> bigbang-podinfo bigbang
- ```
+ ```shell
+ # Iron Bank credentials are optional until we migrate the package to an Iron Bank image
+ helm upgrade -i -n bigbang --create-namespace -f ~/bigbang/chart/values.yaml -f ~/bigbang/chart/ingress-certs.yaml -f bigbang/values.yaml --set registryCredentials.username=<your Iron Bank username> --set registryCredentials.password=<your Iron Bank PAT> bigbang-podinfo bigbang
+ ```
1. Watch the `GitRepository`, `HelmRelease`, and `Pods` by running `watch kubectl get gitrepo,hr,po -A`. Istio operator, Istio control plane, and the package should all be installed.
@@ -225,112 +225,112 @@ Test the following items to ensure Istio is working properly with your applicati
1. Validate the TLS certificate is in the secret `istio-system/public-cert`. This is the certificate Istio uses to provide https access to your package from outside the cluster.
- ```shell
- kubectl get secret -n istio-system public-cert -o 'go-template={{ index .data "tls.crt" }}' | base64 -d
+ ```shell
+ kubectl get secret -n istio-system public-cert -o 'go-template={{ index .data "tls.crt" }}' | base64 -d
- ## OUTPUT ##
- -----BEGIN CERTIFICATE-----
- MIIFYDCCB...
- ```
+ ## OUTPUT ##
+ -----BEGIN CERTIFICATE-----
+ MIIFYDCCB...
+ ```
- > Istio self generates a different certificate for mutual TLS encryption between the Istio control plane and the Istio sidecar inside the pod for traffic into the pod.
+ > Istio self generates a different certificate for mutual TLS encryption between the Istio control plane and the Istio sidecar inside the pod for traffic into the pod.
1. Validate Istio's Ingress Gateway service is running. This is the load balancer that listens for traffic coming from outside the cluster.
- ```shell
- kubectl describe services -n istio-system public-ingressgateway
+ ```shell
+ kubectl describe services -n istio-system public-ingressgateway
- ## OUTPUT ##
- Name: public-ingressgateway
- Namespace: istio-system
- Labels: ...
- ```
+ ## OUTPUT ##
+ Name: public-ingressgateway
+ Namespace: istio-system
+ Labels: ...
+ ```
- > Big Bang defaults to a single ingress gateway called `public-ingressgateway`.
+ > Big Bang defaults to a single ingress gateway called `public-ingressgateway`.
- In the description, you can see the following:
- - Type is `LoadBalancer`
- - `status-port` is configured for a health status check on ingress
- - `http` is configured to listen on port 80 and forward to port 8080
- - `https` is configured to listen on port 443 and forward to port 8443
- - LoadBalancer Ingress IPs are external IPs assigned to the load balancer that can be accessed outside of the cluster.
+ In the description, you can see the following:
+ - Type is `LoadBalancer`
+ - `status-port` is configured for a health status check on ingress
+ - `http` is configured to listen on port 80 and forward to port 8080
+ - `https` is configured to listen on port 443 and forward to port 8443
+ - LoadBalancer Ingress IPs are external IPs assigned to the load balancer that can be accessed outside of the cluster.
1. Validate Istio's Gateway is configured correctly. Istio will use the Gateway's configuration for TLS encryption and host/port-based traffic matching.
- ```shell
- kubectl describe gateway -n istio-system
+ ```shell
+ kubectl describe gateway -n istio-system
- ## OUTPUT ##
- Name: public
- Namespace: istio-system
- Labels: ...
- ```
+ ## OUTPUT ##
+ Name: public
+ Namespace: istio-system
+ Labels: ...
+ ```
- > Big Bang defaults to a single `Gateway` called `public`.
+ > Big Bang defaults to a single `Gateway` called `public`.
- By default the Gateway is setup with the following:
+ By default the Gateway is setup with the following:
- - Connected to the `public-ingressgateway` using `spec.selector`
- - `http`:
- - Matches traffic for any hostname
- - Listen on port 8080
- - Redirects all traffic to `https`
- - `https`:
- - Matches all hosts in domain (`*.bigbang.dev`)
- - Listen on port 8443
- - Uses TLS certificate from `public-cert` for encryption/decryption
+ - Connected to the `public-ingressgateway` using `spec.selector`
+ - `http`:
+ - Matches traffic for any hostname
+ - Listen on port 8080
+ - Redirects all traffic to `https`
+ - `https`:
+ - Matches all hosts in domain (`*.bigbang.dev`)
+ - Listen on port 8443
+ - Uses TLS certificate from `public-cert` for encryption/decryption
1. Validate the package's virtual service. The virtual service controls the traffic routing between Istio and the package.
- ```shell
- kubectl describe virtualservice -n podinfo
+ ```shell
+ kubectl describe virtualservice -n podinfo
- ## OUTPUT ##
- Name: podinfo-podinfo
- Namespace: podinfo
- Labels: ...
- ```
+ ## OUTPUT ##
+ Name: podinfo-podinfo
+ Namespace: podinfo
+ Labels: ...
+ ```
- The virtual service should have the following configured:
- - Connection to Istio's `istio-system/public` Gateway using `spec.gateway`
- - Matches specific host for package (`podinfo.bigbang.dev`)
- - Routes `http` traffic to package's service and port. You can view the service name and `http` port using `kubectl describe service -n podinfo`.
+ The virtual service should have the following configured:
+ - Connection to Istio's `istio-system/public` Gateway using `spec.gateway`
+ - Matches specific host for package (`podinfo.bigbang.dev`)
+ - Routes `http` traffic to package's service and port. You can view the service name and `http` port using `kubectl describe service -n podinfo`.
1. Validate Istio's sidecar is running in the package's pod.
- ```shell
- # Get "ready" status of running containers
- kubectl get pods -n podinfo -o jsonpath='{range .items[*].status.containerStatuses[*]}{@.name}{" "}{@.ready}{"\n"}'
+ ```shell
+ # Get "ready" status of running containers
+ kubectl get pods -n podinfo -o jsonpath='{range .items[*].status.containerStatuses[*]}{@.name}{" "}{@.ready}{"\n"}'
- ## OUTPUT ##
- istio-proxy true
- podinfo true
- ```
+ ## OUTPUT ##
+ istio-proxy true
+ podinfo true
+ ```
- > If `istio-proxy` is not listed in the running containers, try restarting your pod. Use `kubectl get deploy,sts,ds -n podinfo` to get the name and `kubectl rollout restart -n podinfo <name>` to restart it. Then, check for the `istio-proxy` container again.
+ > If `istio-proxy` is not listed in the running containers, try restarting your pod. Use `kubectl get deploy,sts,ds -n podinfo` to get the name and `kubectl rollout restart -n podinfo <name>` to restart it. Then, check for the `istio-proxy` container again.
1. Check the package's URL
- > The hostname `*.bigbang.dev` points to your local IP address of 127.0.0.1. If you are running the cluster on a different machine, you will need to add the hostname and host machine's IP to `/etc/hosts`.
-
- ```shell
- curl -sL https://podinfo.bigbang.dev
-
- ## OUTPUT ##
- {
- "hostname": "podinfo-podinfo-86b4b9d85c-rnd4z",
- "version": "6.0.0",
- "revision": "",
- "color": "#34577c",
- "logo": "https://raw.githubusercontent.com/stefanprodan/podinfo/gh-pages/cuddle_clap.gif",
- "message": "greetings from podinfo v6.0.0",
- "goos": "linux",
- "goarch": "amd64",
- "runtime": "go1.15.7",
- "num_goroutine": "8",
- "num_cpu": "32"
- }
- ```
+ > The hostname `*.bigbang.dev` points to your local IP address of 127.0.0.1. If you are running the cluster on a different machine, you will need to add the hostname and host machine's IP to `/etc/hosts`.
+
+ ```shell
+ curl -sL https://podinfo.bigbang.dev
+
+ ## OUTPUT ##
+ {
+ "hostname": "podinfo-podinfo-86b4b9d85c-rnd4z",
+ "version": "6.0.0",
+ "revision": "",
+ "color": "#34577c",
+ "logo": "https://raw.githubusercontent.com/stefanprodan/podinfo/gh-pages/cuddle_clap.gif",
+ "message": "greetings from podinfo v6.0.0",
+ "goos": "linux",
+ "goarch": "amd64",
+ "runtime": "go1.15.7",
+ "num_goroutine": "8",
+ "num_cpu": "32"
+ }
+ ```
You have now verified Istio is working properly with the package. To recap, incoming traffic to the cluster is first processed by Istio's ingress gateway listening on specific ports on each node. `http` and `https` traffic is forwarded to internal ports 8080 and 8443 respectively. The Istio Gateway configuration redirects `http` traffic to `https` and `https` traffic matching the domain (`bigbang.dev`) is TLS decrypted. The Virtual Service configuration processes `https` traffic from the Gateway matching the package's hostname (`podinfo.bigbang.dev`) and routes traffic to the package's service and `http` port. The service then directs traffic to the pod for handling. Since the pod has the Istio sidecar running, the mutual TLS Istio feature will automatically encrypt traffic from the gateway to the pod inside the cluster (even though its http). The sidecar will then decrypt to package before sending it over to the package for handling. The following diagram illustrates this flow:
diff --git a/docs/developer/package-integration/sso.md b/docs/developer/package-integration/sso.md
index 3b427a495c6ca61da315e951b5cf306f1d9533b0..2ae76a9dc2e4ac5aa2579ad4e9399d1be456cdfd 100644
--- a/docs/developer/package-integration/sso.md
+++ b/docs/developer/package-integration/sso.md
@@ -57,19 +57,19 @@ In order to use Authservice, Istio injection is required and utilized to route a
1. The first step is to ensure your namespace template where you package is destined is istio injected, and the appropriate label is set in `chart/templates/<package>/namespace.yaml`.
-Example: [Jaeger Namespace template](../../../chart/templates/jaeger/namespace.yaml)
+ Example: [Jaeger Namespace template](../../../chart/templates/jaeger/namespace.yaml)
1. Next is to make sure the following label is applied to the workload (pod/deployment/replicaset/daemonset/etc) that will be behind the Authservice gate:
-```yaml
-...
-{{- $<package>AuthserviceKey := (dig "selector" "key" "protect" .Values.addons.authservice.values) }}
-{{- $<package>AuthserviceValue := (dig "selector" "value" "keycloak" .Values.addons.authservice.values) }}
-...
-metadata:
- labels:
- {{ $<package>AuthserviceKey }}: {{ $<package>AuthserviceValue }}
-```
+ ```yaml
+ ...
+ {{- $<package>AuthserviceKey := (dig "selector" "key" "protect" .Values.addons.authservice.values) }}
+ {{- $<package>AuthserviceValue := (dig "selector" "value" "keycloak" .Values.addons.authservice.values) }}
+ ...
+ metadata:
+ labels:
+ {{ $<package>AuthserviceKey }}: {{ $<package>AuthserviceValue }}
+ ```
This label is set in the Authservice package, and is set to `protect=keycloak` by default, the above logic will check if anyone overwrites these values within their BigBang installation and overwrite the label accordingly.
diff --git a/docs/developer/package-integration/supported.md b/docs/developer/package-integration/supported.md
index 4a6101e10a14756a773bb82c2d43c29c0facdeaa..20bcf37d1cb9d2032cbca4d382cc0fc4779fa1cc 100644
--- a/docs/developer/package-integration/supported.md
+++ b/docs/developer/package-integration/supported.md
@@ -32,37 +32,37 @@ After [graduating your package](https://repo1.dso.mil/platform-one/bbtoc/-/tree/
1. Merge your default package values from `<your-package-git-folder>/bigbang/values.yaml` into `chart/values.yaml`. Only the "standard" keys used across packages should be used. Keep in mind that values can be passed directly to the package using `.Values.<package>.values`
- > If your package is an `addon`, it falls into a different location than core packages. In this case, you will need to update all your references from `.Values.<package>` to `.Values.addons.<package>`.
-
- Example:
-
- ```yaml
- addons:
- mypackage:
- enabled: false # default to false
- git:
- repo: https://repo1.dso.mil/platform-one/big-bang/apps/developer-tools/mypackage.git
- path: "./chart"
- tag: "1.2.3-bb.0"
- sso:
- enabled: false # default to false
- client_id: ""
- database:
- host: ""
- port: ""
- username: ""
- database: ""
- password: {} # unencoded stringData
- objectstorage:
- type: s3 # supported types are "s3" or "minio"
- endpoint: "" # ignored if type is "s3". used only for minio. example " http://minio.minio.svc.cluster.local:9000"
- host: s3.amazonaws.com # used for gitlab backup storage
- region: us-west-1
- accessKey: ""
- accessSecret: ""
- bucketPrefix: "" # optional. example: dev-
- values: {}
- ```
+ > If your package is an `addon`, it falls into a different location than core packages. In this case, you will need to update all your references from `.Values.<package>` to `.Values.addons.<package>`.
+
+ Example:
+
+ ```yaml
+ addons:
+ mypackage:
+ enabled: false # default to false
+ git:
+ repo: https://repo1.dso.mil/platform-one/big-bang/apps/developer-tools/mypackage.git
+ path: "./chart"
+ tag: "1.2.3-bb.0"
+ sso:
+ enabled: false # default to false
+ client_id: ""
+ database:
+ host: ""
+ port: ""
+ username: ""
+ database: ""
+ password: {} # unencoded stringData
+ objectstorage:
+ type: s3 # supported types are "s3" or "minio"
+ endpoint: "" # ignored if type is "s3". used only for minio. example " http://minio.minio.svc.cluster.local:9000"
+ host: s3.amazonaws.com # used for gitlab backup storage
+ region: us-west-1
+ accessKey: ""
+ accessSecret: ""
+ bucketPrefix: "" # optional. example: dev-
+ values: {}
+ ```
1. Edit `tests/test-values.yaml`. These are the settings that the CI pipeline uses to run a deployment test. Set your Package to be enabled and add any other necessary values. Where possible reduce the number of replicas to a minimum to reduce strain on the CI infrastructure. When you commit your code the pipeline will run. You can view the pipeline in the Repo1 Gitlab console. Fix any errors in the pipeline output. The pipeline automatically runs a "smoke" test. It deploys bigbang on a k3d cluster using the test values file.
diff --git a/docs/developer/package-integration/upstream.md b/docs/developer/package-integration/upstream.md
index 7d540051657eb06fef48b1662bb25aa6ac016dae..e7fd0fdbb6e47ab74e0e91b7589b1ff0b98ed9d4 100644
--- a/docs/developer/package-integration/upstream.md
+++ b/docs/developer/package-integration/upstream.md
@@ -27,35 +27,35 @@ To minimize maintenance, it is preferable to reuse existing Helm charts availabl
1. Identify the location of an existing Helm chart for the package.
- > If selecting between several Helm charts, give preference to a Helm chart that:
- >
- > - Was created by the company that owns the package
- > - Has recent and frequent updates
- > - Offers maximum flexibility through values
- > - Does not bundle several packages together (unless they can be individually disabled)
- > - Provides advanced features like high availability, scaling, affinity, taints/tolerations, and security context
+ > If selecting between several Helm charts, give preference to a Helm chart that:
+ >
+ > - Was created by the company that owns the package
+ > - Has recent and frequent updates
+ > - Offers maximum flexibility through values
+ > - Does not bundle several packages together (unless they can be individually disabled)
+ > - Provides advanced features like high availability, scaling, affinity, taints/tolerations, and security context
1. Using [Kpt](https://googlecontainertools.github.io/kpt/installation/), create a clone of the package's Helm chart
- ```shell
- # Change these for your upstream helm chart
- export GITREPO=https://github.com/stefanprodan/podinfo
- export GITDIR=charts/podinfo
- export GITTAG=5.2.1
+ ```shell
+ # Change these for your upstream helm chart
+ export GITREPO=https://github.com/stefanprodan/podinfo
+ export GITDIR=charts/podinfo
+ export GITTAG=5.2.1
- # Clone
- kpt pkg get $GITREPO/$GITDIR@$GITTAG chart
- ```
+ # Clone
+ kpt pkg get $GITREPO/$GITDIR@$GITTAG chart
+ ```
- > Always use an release tag for `GITTAG` so your chart is immutable. Never use a branch or `latest`.
+ > Always use an release tag for `GITTAG` so your chart is immutable. Never use a branch or `latest`.
1. Add `-bb.0` suffix on `chart/Chart.yaml`, `version`. For example:
- ```yaml
- version: 6.0.0-bb.0
- ```
+ ```yaml
+ version: 6.0.0-bb.0
+ ```
- > The `bb.#` will increment for each change we merge into our `main` branch. It will also become our release label.
+ > The `bb.#` will increment for each change we merge into our `main` branch. It will also become our release label.
1. Add the following files to the Git repository at the root:
diff --git a/docs/guides/deployment-scenarios/quickstart.md b/docs/guides/deployment-scenarios/quickstart.md
index 1d7e96516dc91904d707a98c6029d5b4e1eeff2c..b9537368714163a8c48d72ac52b8464bea168d07 100644
--- a/docs/guides/deployment-scenarios/quickstart.md
+++ b/docs/guides/deployment-scenarios/quickstart.md
@@ -894,19 +894,21 @@ After completing step 5, if you are unable to connect to external DNS providers
```shell
kubectl -n kube-system edit configmaps CoreDNS -o yaml
```
-2. Change:
- ```plaintext
- forward . /etc/resolv.conf
- ```
+1. Change:
- To:
+ ```plaintext
+ forward . /etc/resolv.conf
+ ```
- ```plaintext
- forward . <DNS Server IP>
- ```
-3. Save changes in editor (for vi use `:wq`)
-4. Verify changes in terminal output that prints new config
+ To:
+
+ ```plaintext
+ forward . <DNS Server IP>
+ ```
+
+1. Save changes in editor (for vi use `:wq`)
+1. Verify changes in terminal output that prints new config
### Useful Commands for Obtaining Detailed Logs from Kubernetes Cluster or Containers
* Print all pods including information related to the status of each pod
diff --git a/docs/prerequisites/kubernetes-preconfiguration.md b/docs/prerequisites/kubernetes-preconfiguration.md
index 78fdc773fc5b60f5bb6681d23c4cb0f481fd165e..2dbed7a43c650041485d73fd56d83efe38eba31a 100644
--- a/docs/prerequisites/kubernetes-preconfiguration.md
+++ b/docs/prerequisites/kubernetes-preconfiguration.md
@@ -63,9 +63,7 @@ It is important to note that while Big Bang does not require/mandate usage of a
### OpenShift
-OpenShift
-
-1) When deploying BigBang, set the OpenShift flag to true.
+1. When deploying BigBang, set the OpenShift flag to true.
```yaml
# inside a values.yaml being passed to the command installing bigbang
@@ -77,14 +75,15 @@ OpenShift
helm install bigbang chart --set openshift=true
```
-1) Patch the istio-cni daemonset to allow containers to run privileged (AFTER istio-cni daemonset exists).
-Note: it was unsuccessfully attempted to apply this setting via modifications to the helm chart. Online patching succeeded.
+1. Patch the istio-cni daemonset to allow containers to run privileged (AFTER istio-cni daemonset exists).
+
+ Note: it was unsuccessfully attempted to apply this setting via modifications to the helm chart. Online patching succeeded.
```shell
kubectl get daemonset istio-cni-node -n kube-system -o json | jq '.spec.template.spec.containers[] += {"securityContext":{"privileged":true}}' | kubectl replace -f -
```
-1) Modify the OpenShift cluster(s) with the following scripts based on <https://istio.io/v1.7/docs/setup/platform-setup/openshift/>
+1. Modify the OpenShift cluster(s) with the following scripts based on <https://istio.io/v1.7/docs/setup/platform-setup/openshift/>
```shell
# Istio Openshift configurations Post Install
diff --git a/docs/understanding-bigbang/concepts/deployment.md b/docs/understanding-bigbang/concepts/deployment.md
index 0c6bc984dee71b72542484e92e39edbb13227716..eda8043a41a108008a22ed6109e272c31c31ec5f 100644
--- a/docs/understanding-bigbang/concepts/deployment.md
+++ b/docs/understanding-bigbang/concepts/deployment.md
@@ -10,29 +10,29 @@ Big Bang follows a [GitOps](https://www.weave.works/blog/what-is-gitops-really)
1. Before pushing changes to Git, validate all configuration is syntactically correct.
- ```shell
- # If everything is successful, YAML should be output
- kustomize build ./dev
- ```
+ ```shell
+ # If everything is successful, YAML should be output
+ kustomize build ./dev
+ ```
1. If you have not already done so, push configuration changes to Git
- ```shell
- git push
- ```
+ ```shell
+ git push
+ ```
1. Validate the Kubernetes context is correct
- ```shell
- # This should match the environment you intend to deploy
- kubectl config current-context
- ```
+ ```shell
+ # This should match the environment you intend to deploy
+ kubectl config current-context
+ ```
1. Deploy the Big Bang manifest to the cluster
- ```shell
- kubectl apply -f dev.yaml
- ```
+ ```shell
+ kubectl apply -f dev.yaml
+ ```
1. [Monitor the deployment](#monitor)
@@ -50,86 +50,86 @@ The following commands will help you monitor the progress of the Big Bang deploy
1. Verify Flux is running
- ```shell
- kubectl get deploy -n flux-system
+ ```shell
+ kubectl get deploy -n flux-system
- # All resources should be in the 'Ready' state
- NAME READY UP-TO-DATE AVAILABLE AGE
- source-controller 1/1 1 1 106s
- kustomize-controller 1/1 1 1 106s
- notification-controller 1/1 1 1 105s
- helm-controller 1/1 1 1 106s
- ```
+ # All resources should be in the 'Ready' state
+ NAME READY UP-TO-DATE AVAILABLE AGE
+ source-controller 1/1 1 1 106s
+ kustomize-controller 1/1 1 1 106s
+ notification-controller 1/1 1 1 105s
+ helm-controller 1/1 1 1 106s
+ ```
1. Verify the environment was pulled from the Git repo
- ```shell
- kubectl get gitrepository -A
+ ```shell
+ kubectl get gitrepository -A
- # `environment-repo`: STATUS should be True
- NAMESPACE NAME URL READY STATUS AGE
- bigbang environment-repo https://repo1.dso.mil/platform-one/big-bang/customers/template.git True Fetched revision: main/185e252f4452d897531ab0314adc7a189562be31 2m7s
- ```
+ # `environment-repo`: STATUS should be True
+ NAMESPACE NAME URL READY STATUS AGE
+ bigbang environment-repo https://repo1.dso.mil/platform-one/big-bang/customers/template.git True Fetched revision: main/185e252f4452d897531ab0314adc7a189562be31 2m7s
+ ```
1. Verify the environment Kustomization properly worked
- ```shell
- kubectl get kustomizations -A
+ ```shell
+ kubectl get kustomizations -A
- # `environment`: READY should be True
- NAMESPACE NAME READY STATUS AGE
- bigbang environment True Applied revision: main/185e252f4452d897531ab0314adc7a189562be31 6m41s
- ```
+ # `environment`: READY should be True
+ NAMESPACE NAME READY STATUS AGE
+ bigbang environment True Applied revision: main/185e252f4452d897531ab0314adc7a189562be31 6m41s
+ ```
1. Verify the ConfigMaps were deployed
- ```shell
- kubectl get configmap -l kustomize.toolkit.fluxcd.io/namespace -A
+ ```shell
+ kubectl get configmap -l kustomize.toolkit.fluxcd.io/namespace -A
- # 'common' and 'environment' should exist
- NAMESPACE NAME DATA AGE
- bigbang common-cch6942dk9 1 19m
- bigbang environment-d2tgb27f56 1 19m
- ```
+ # 'common' and 'environment' should exist
+ NAMESPACE NAME DATA AGE
+ bigbang common-cch6942dk9 1 19m
+ bigbang environment-d2tgb27f56 1 19m
+ ```
1. Verify the Secrets were deployed
- ```shell
- kubectl get secrets -l kustomize.toolkit.fluxcd.io/namespace -A
+ ```shell
+ kubectl get secrets -l kustomize.toolkit.fluxcd.io/namespace -A
- # 'common-bb' and 'environment-bb' should exist
+ # 'common-bb' and 'environment-bb' should exist
NAMESPACE NAME TYPE DATA AGE
bigbang common-bb-kc5t8dbdfh Opaque 1 18m
bigbang environment-bb-mhddkt46bd Opaque 1 18m
- ```
+ ```
1. Verify the Big Bang Helm Chart was pulled
- ```shell
- kubectl get gitrepositories -A
+ ```shell
+ kubectl get gitrepositories -A
- # 'bigbang' READY should be True
+ # 'bigbang' READY should be True
NAME URL READY STATUS AGE
bigbang https://repo1.dso.mil/platform-one/big-bang/bigbang.git True Fetched revision: master/8a4a1ddd0c9edf316f5362680cf2921baf0c3451 25m
- ```
+ ```
1. Verify the Big Bang Helm Chart was deployed
- ```shell
- kubectl get hr -A
+ ```shell
+ kubectl get hr -A
- # 'bigbang' READY should be True
+ # 'bigbang' READY should be True
NAMESPACE NAME READY STATUS AGE
bigbang bigbang True Release reconciliation succeeded 28m
- ```
+ ```
1. Verify Big Bang package Helm charts are pulled
- ```shell
- kubectl get gitrepository -A
+ ```shell
+ kubectl get gitrepository -A
- # The Git repository holding the Helm charts for each package can be seen in the URL column.
- # The STATUS column shows the branch and tag of the revision being used.
+ # The Git repository holding the Helm charts for each package can be seen in the URL column.
+ # The STATUS column shows the branch and tag of the revision being used.
NAMESPACE NAME URL READY STATUS AGE
bigbang bigbang https://repo1.dso.mil/platform-one/big-bang/bigbang.git True Fetched revision: master/3a44686520152e576a8c2c6f264876efff497c4b 8m25s
bigbang logging https://repo1.dso.mil/platform-one/big-bang/apps/core/logging.git True Fetched revision: release-v0.2.x/9cfe1e14c12098464ee89eb877614f781cd78fb7 8m23s
@@ -139,16 +139,16 @@ The following commands will help you monitor the progress of the Big Bang deploy
bigbang gatekeeper https://repo1.dso.mil/platform-one/big-bang/apps/core/policy.git True Fetched revision: chart-release/1a5f32c8e7f672c3b5937b604e5f38eaa08ce246 8m23s
bigbang monitoring https://repo1.dso.mil/platform-one/big-bang/apps/core/monitoring.git True Fetched revision: release-v0.2.x/ca60bedcc106b95beb0bf9ccdc6e0e759e6fd6bf 8m23s
bigbang cluster-auditor https://repo1.dso.mil/platform-one/big-bang/apps/core/cluster-auditor.git True Fetched revision: chart-release/598c35670db0cbdb3a48063b2d558965afe73185 8m23s
- ```
+ ```
1. Verify the packages get deployed
- ```shell
- # Use watch since it take a long time to deploy
- watch kubectl get hr,deployments,po -A
+ ```shell
+ # Use watch since it take a long time to deploy
+ watch kubectl get hr,deployments,po -A
- # Flux will not attempt to deploy a package until its dependencies are ready
- # All Helm Release resources and Pods
+ # Flux will not attempt to deploy a package until its dependencies are ready
+ # All Helm Release resources and Pods
Every 2.0s: kubectl get hr,deployments,po -A localhost: Mon Nov 9 10:14:56 2020
NAMESPACE NAME READY STATUS AGE
@@ -186,8 +186,8 @@ The following commands will help you monitor the progress of the Big Bang deploy
gatekeeper-system pod/gatekeeper-controller-manager-5b9cf6c85d-cqd8t 1/1 Running 0 2m8s
gatekeeper-system pod/gatekeeper-audit-7db49c54d5-pwzwh 1/1 Running 0 2m8s
istio-operator pod/istio-operator-79f966cfc-rjhhc 0/1 ContainerCreating 0 8s
- ```
+ ```
1. Wait until all Helm Releases, Deployments, and Pods are Ready. Be patient, this can take 15-30 minutes.
-> The Git repositories are monitored periodically (default is 10m) for changes. If a change is detected, the configuration will be reconciled using Flux. The monitoring techniques above can be used to monitor the reconciliation.
+ > The Git repositories are monitored periodically (default is 10m) for changes. If a change is detected, the configuration will be reconciled using Flux. The monitoring techniques above can be used to monitor the reconciliation.