diff --git a/docs/developer/package-integration/package-integration-flux.md b/docs/developer/package-integration/package-integration-flux.md index c472c63417833486b1cf419ae1f0ad9a2bab32e8..64373851b7320a0be0e7fd596dea8bd913d80a54 100644 --- a/docs/developer/package-integration/package-integration-flux.md +++ b/docs/developer/package-integration/package-integration-flux.md @@ -4,8 +4,370 @@ Big Bang uses a continuous deployment tool, [Flux](https://fluxcd.io/) to deploy ## Prerequisites -TBD +- [Helm](https://helm.sh/docs/intro/install/) +- [Kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl) +- [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) +- A multi-node Kubernetes cluster to deploy Big Bang and your package +- A [Big Bang project containing the upstream Helm chart](./package-integration-upstream.md) -## Integration +> Throughout this document, we will be setting up an application called `podinfo` as a demonstration -## Validation +## Big Bang Helm Chart + +The purpose of the Big Bang Helm chart is to create a Big Bang compatible, easy-to-use spec for deploying the package. Reasonable and safe defaults are provided and any needed secrets are auto-created. We accept the trade-off of easy deployment for complicated template code. Details are in the following steps. + + ```shell + gitrepository.yaml # Flux GitRepository, configured by Big Bang chart values. + helmrelease.yaml # Flux HelmRelease, configured by Big Bang chart values. + namespace.yaml # Namespace creation and configuration + imagepullsecret.yaml # Secret creation for image pull credentials + values.yaml # Big Bang customization of the package and passthrough values. + ``` + +Create a new Helm chart for Big Bang resources in the root of your Git repository: + +```shell +# short name of the package +export PKGNAME=podinfo + +# version of the package in semver format +export PKGVER=6.0.0 + +# Make directory structure +mkdir -p bigbang/templates/$PKGNAME + +# Create values file +touch bigbang/values.yaml + +# Copy helpers from Big Bang +curl -sL -o bigbang/templates/_helpers.tpl https://repo1.dso.mil/platform-one/big-bang/bigbang/-/raw/master/chart/templates/_helpers.tpl + +# Create chart file +cat << EOF >> bigbang/Chart.yaml +apiVersion: v2 +name: bigbang-$PKGNAME +description: BigBang compatible Helm chart for $PKGNAME +type: application +version: 0.1.0 +appVersion: "$PKGVER" +EOF +``` + +### Namespace + +The package will be deployed in its own namespace. BigBang pre-creates this namespace so that labels and annotations can be controlled. Setup `bigbang/templates/$PKGNAME/namespace.yaml` with the following: + +```yaml +{{- $pkg := "podinfo" }} +{{- if (get .Values $pkg).enabled }} +apiVersion: v1 +kind: Namespace +metadata: + name: {{ $pkg }} + labels: + app.kubernetes.io/name: {{ $pkg }} + {{- include "commonLabels" . | nindent 4}} +{{- end }} +``` + +In order for the namespace Helm template to be properly created, the following values need to be added to `bigbang/values.yaml`: + +```yaml +# Identifies if our package should be deployed or ignored +podinfo: + enabled: true +``` + +### Flux Custom Resources + +#### GitRepository + +Flux's source controller uses the [GitRepository](https://fluxcd.io/docs/components/source/gitrepositories/) resource to pull Helm chart changes from Git. Use the [GitRepository API Specification](https://fluxcd.io/docs/components/source/gitrepositories/#specification) to create a `GitRepository` resource named `bigbang/templates/$PKGNAME/gitrepository.yaml` with the following content: + +```yaml +{{- $pkg := "podinfo" }} +{{- if (get .Values $pkg).enabled }} +apiVersion: source.toolkit.fluxcd.io/v1beta1 +kind: GitRepository +metadata: + name: {{ $pkg }} + namespace: {{ .Release.Namespace }} + labels: + app.kubernetes.io/name: {{ $pkg }} + {{- include "commonLabels" . | nindent 4}} +spec: + interval: {{ .Values.flux.interval }} + url: {{ (get .Values $pkg).git.repo }} + ref: + {{- include "validRef" (get .Values $pkg).git | nindent 4 }} + {{ include "gitIgnore" . }} + {{- include "gitCreds" . | nindent 2 }} +{{- end }} +``` + +The `GitRepository` Helm template above requires the following values to be added to `flx/values.yaml`: + +```yaml +podinfo: + # The Git location of the package Helm chart + git: + repo: https://repo1.dso.mil/platform-one/big-bang/apps/sandbox/podinfo + branch: master +``` + +> If you are working on a branch, change `master` to the branch you are working from in the values above. + +#### HelmRelease + +Big Bang exclusively uses Helm charts for deployment through Flux. Using the [HelmRelease API Specification](https://fluxcd.io/docs/components/helm/helmreleases/#specification), create a `HelmRelease` resource named `bigbang/templates/$PKGNAME/helmrelease.yaml` with the following content: + +```yaml +{{- $pkg := "podinfo" }} +{{- $fluxSettings := merge (get .Values $pkg).flux .Values.flux -}} +{{- if (get .Values $pkg).enabled }} +apiVersion: helm.toolkit.fluxcd.io/v2beta1 +kind: HelmRelease +metadata: + name: {{ $pkg }} + namespace: {{ .Release.Namespace }} + labels: + app.kubernetes.io/name: {{ $pkg }} + {{- include "commonLabels" . | nindent 4}} +spec: + targetNamespace: {{ $pkg }} + chart: + spec: + chart: {{ (get .Values $pkg).git.path }} + interval: 5m + sourceRef: + kind: GitRepository + name: {{ $pkg }} + namespace: {{ .Release.Namespace }} + + {{- toYaml $fluxSettings | nindent 2 }} + + {{- if (get .Values $pkg).postRenderers }} + postRenderers: + {{ toYaml (get .Values $pkg).postRenderers | nindent 4 }} + {{- end }} + valuesFrom: + - name: {{ .Release.Name }}-{{ $pkg }}-values + kind: Secret + valuesKey: "common" + - name: {{ .Release.Name }}-{{ $pkg }}-values + kind: Secret + valuesKey: "defaults" + - name: {{ .Release.Name }}-{{ $pkg }}-values + kind: Secret + valuesKey: "overlays" +{{- end }} +``` + +The following values need to be added into `bigbang/values.yaml` for `HelmRelease`: + +```yaml +podinfo: + # Directory in git where Helm chart is located + git: + path: chart + # Flux specific settings for package + flux: {} +``` + +### ImagePullSecret + +Big Bang images are pulled from Iron Bank. In order to provide credentials for Iron Bank, Big Bang will create a secret for each package called `private-registry`. In `bigbang/templates/$PKGNAME/imagepullsecret.yaml`, add the following content: + +```yaml +{{- $pkg := "podinfo" }} +{{- if (get .Values $pkg).enabled }} +{{- if ( include "imagePullSecret" . ) }} +apiVersion: v1 +kind: Secret +metadata: + name: private-registry + namespace: {{ $pkg }} + labels: + app.kubernetes.io/name: {{ $pkg }} + {{- include "commonLabels" . | nindent 4}} +type: kubernetes.io/dockerconfigjson +data: + .dockerconfigjson: {{ template "imagePullSecret" . }} +{{- end }} +{{- end }} +``` + +> Other secrets can be added for credentials, certificates, etc. by creating a file named `bigbang/templates/$PKGNAME/secret-<name>.yaml`. Big Bang is responsible for creating these secrets using values from the user. More details are included in the integration documentation for databases, object stores, sso, etc. + +### Package Values + +Package values (chart/values.yaml) should contain upstream values plus any placeholders of values needed for Big Bang.The following guidelines should be used when adding values to the package: + +- Assume the package will be run without Big Bang. Values enabling features from other packages (e.g. metrics, ingress, SSO) should be turned off by default. Big Bang will enable them through overrides. +- Re-use existing chart values rather than adding new ones when possible. +- Only change the default values from the upstream Helm chart when necessary. +- Comment any changes made to the upstream Helm values so it is clear that the changes should carry forward on upgrades. +- Assume that Big Bang will create secrets (e.g. TLS certificates, credentials) and provide the reference to the chart. +- Create blank placeholders for Big Bang values to avoid Helm errors during deployment. + +### Override Values + +Big Bang has a few options for overwriting values in packages. The package's `HelmRelease`, that we created earlier, contains a `ValuesFrom` section that references a secret with `common`, `default`, and `overlay` keys. Each of these keys can contain a set of override values that get passed down to the package. Here is a table explaining the difference between the possible overlays: + +|Name|Description|Source|Priority| +|--|--|--|--| +| `overlay` | Values provided by user when deploying Big Bang | `bigbang/values.yaml`:`$PKGNAME.values.*` | Highest 1 | +| `default` | Values created by Big Bang | `bigbang/templates/$PKGNAME/values.yaml`:`*` | 2 | +| `common` | Big Bang values common to all packages | Not currently used | 3 | +| `package` | Package defaults | `chart/values.yaml`:`*` | Lowest 4 | + +This means that if a user provides a value for the package, that overwrites the value Big Bang or the package would create. + +For the package to implement this hierarchy, `bigbang/templates/$PKGNAME/values.yaml` must be created with the following: + +```yaml +{{- $pkg := "podinfo" }} +{{- define "bigbang.defaults.podinfo" -}} + +{{- end }} + +{{- /* Create secret */ -}} +{{- if (get .Values $pkg).enabled }} +{{- include "values-secret" (dict "root" $ "package" (get .Values $pkg) "name" $pkg "defaults" (include (printf "bigbang.defaults.%s" $pkg) .)) }} +{{- end }} +``` + +### Check Syntax + +At this point, you should have a minimum viable set of values in `bigbang/values.yaml` that looks like this: + +```yaml +podinfo: + enabled: true + git: + repo: https://repo1.dso.mil/platform-one/big-bang/apps/sandbox/podinfo + branch: bigbang + path: chart + flux: {} +``` + +Use the Big Bang default values to make sure our Helm templates don't have any syntax errors. Run the following: + + ```shell + # Get the helm chart + git clone https://repo1.dso.mil/platform-one/big-bang/bigbang ~/bigbang + + # Check that our chart generates without errors + # We want our local values to override the big bang defaults, so we need to specify both + helm template -n bigbang -f ~/bigbang/chart/values.yaml -f bigbang/values.yaml bigbang-podinfo bigbang + ``` + +### Validation + +To validate that the Helm chart is working, perform the following steps to deploy your package. This assumes you already have a Kubernetes cluster running. + +1. Disable all default packages in Big Bang by adding the following to `bigbang/values.yaml` + + ```yaml + # Network Policies + networkPolicies: + enabled: false + + # Istio + istiooperator: + enabled: false + istio: + enabled: false + + # Gatekeeper + gatekeeper: + enabled: false + clusterAuditor: + enabled: false + + # Logging + eckoperator: + enabled: false + logging: + enabled: false + fluentbit: + enabled: false + + # Monitoring + monitoring: + enabled: false + + # Other Tools + jaeger: + enabled: false + kiali: + enabled: false + twistlock: + enabled: false + ``` + +1. Install flux using the [instructions from Big Bang](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/blob/1.19.0/docs/guides/deployment_scenarios/quickstart.md#step-8-install-flux). +1. Install the package using the bigbang Helm chart + + ```shell + helm upgrade -i -n bigbang --create-namespace -f ~/bigbang/chart/values.yaml -f bigbang/values.yaml bigbang-podinfo bigbang + ``` + +1. Watch the `GitRepository`, `HelmRelease`, and `Pods`: + + ```shell + watch kubectl get gitrepo,hr,po -A + ``` + +1. Troubleshoot any errors + + ```shell + kubectl get events -A + ``` + + > If you are using a private Git repository or pulling images from a private image repository, you will need to add credentials into the `git.credentials.username`/`git.credentials.password` and/or `registryCredentials.username`/`registryCredentials.password` using the `--set` option for Helm. + +1. Cleanup cluster + + ```shell + helm delete -n bigbang bigbang-podinfo + ``` + +1. Add the following to `bigbang/README.md` to document this Helm charts usage: + + ```markdown + # Big Bang compatible Helm chart + + This helm chart deploys the application using the same methods and values as Big Bang. + + ## Prerequisites + + - Kubernetes cluster matching [Big Bang's Prerequisites](https://repo1.dso.mil/platform-one/big-bang/bigbang/-/tree/master/docs/guides/prerequisites) + - [FluxCD](https://fluxcd.io/) running in the cluster + - The [Big Bang git repository](https://repo1.dso.mil/platform-one/big-bang/bigbang) cloned into `~/bigbang` + - [Helm](https://helm.sh/docs/intro/install/) + + ## Usage + + ### Installation + + 1. Install Big Bang + `helm upgrade -i -n bigbang --create-namespace -f ~/bigbang/chart/values.yaml -f bigbang/values.yaml bigbang ~/bigbang/chart` + 1. Install this chart + `helm upgrade -i -n bigbang --create-namespace -f ~/bigbang/chart/values.yaml -f bigbang/values.yaml bigbang-podinfo bigbang` + + ### Removal + + `helm delete -n bigbang bigbang-podinfo` + + ``` + +1. Commit your changes + + > If you are developing something different than `podinfo`, run `grep -ir podinfo` to make sure your replaced all of the instances with your application name. + + ```shell + git add -A + git commit -m "feat: added bigbang helm chart" + git push + ```