Release 1.9.0

Release Checklist

1. Release Prep

Verify that the previous release branch commit hash matches the last release tag. Investigate with previous RE if they do not match
Create release branch with name release-1.x.x
Build draft release notes, see release_notes_template.md
Release specific code changes. Make the following changes in a single commit so it can be cherry picked into master later.
- Bump self-reference version in base/gitrepository.yaml
- Update chart release version chart/Chart.yaml
- Update /Packages.md with any new Packages
- Update CHANGELOG.md with links to MRs and any upgrade notices/known issues
- Update README.md using helm-docs. Overwrite the existing readme file.
```
# from root dir of your release branch
docker run -v "$(pwd):/helm-docs" -u $(id -u) jnorwood/helm-docs:v1.5.0 -s file -t .gitlab-ci/README.md.gotmpl --dry-run > README.md
```

2. Test and Validate Release Candidate

Deploy release branch on Dogfood cluster

Connect to Cluster
Update bigbang/base/kustomization.yaml & bigbang/prod/kustomization.yaml with release branch.

Verify cluster has updated to the new release and all package versions match the new chart, flux may take a few minutes to reconcile.

# check release
watch kubectl get gitrepositories,kustomizations,hr,po -A
# if flux has not updated after 10 minutes.
flux reconcile hr -n bigbang bigbang --with-source
# if it is still not updating, delete the flux source controller 
kubectl get all -n flux-system 
kubectl delete pod/source-controller-xxxxxxxx-xxxxx -n flux-system

Confirm app UIs are loading

Logging

Login to kibana with SSO
Kibana is actively indexing/logging.

Cluster Auditor

Login to kibana with SSO
violations index is present and contains images that aren't from registry1

Monitoring

Login to grafana with SSO
Contains Kubernetes Dashboards and metrics
contains istio dashboards

Kiali

Sonarqube

GitLab & Runners

Login to gitlab with SSO
Create new public group with release name. Example release-1-8-0
Create new public project with release name. Example release-1-8-0
git clone and git push to new project

docker push and docker pull image to registry

docker pull alpine
docker tag alpine registry.dogfood.bigbang.dev/GROUPNAMEHERE/PROJECTNAMEHERE/alpine:latest
docker login registry.dogfood.bigbang.dev
docker push registry.dogfood.bigbang.dev/GROUPNAMEHERE/PROJECTNAMEHERE/alpine:latest

Edit profile and change user avatar
Test simple CI pipeline. sample_ci.yaml

Anchore

Login to anchore with SSO
Scan image in dogfood registry, registry.dogfood.bigbang.dev/GROUPNAMEHERE/PROJECTNAMEHERE/alpine:latest
Scan image in registry1, registry.dso.mil/platform-one/big-bang/apps/core/elasticsearch-kibana/busybox

Argocd

Login to argocd with SSO
Logout and login with admin
Create application

Minio

Mattermost

Login to mattermost with SSO
Elastic integration

Velero

Backup PVCs

kubectl apply -f ./velero_test.yaml
# exec into velero_test container
cat /mnt/velero-test/test.log
# take note of log entries and exit exec

velero backup create velero-test-backup-1-8-0 -l app=velero-test
velero backup get
kubectl delete -f ./velero_test.yaml
kubectl get pv | grep velero-test
kubectl delete pv INSERT-PV-ID

Restore PVCs

velero restore create velero-test-restore-1-8-0 --from-backup velero-test-backup-1-8-0
# exec into velero_test container
cat /mnt/velero-test/test.log
# old log entires and new should be in log if backup was done correctly

Keycloak

3. Create Release

Create release candidate tag based on release branch. Tag EX: 1.8.0-rc.0.
```
Message: release canidate
Release Notes: **Leave Blank**
```
Passed tag pipeline.

Create release tag based on release branch. Tag EX: 1.8.0.

Message: release canidate
Release Notes: **Leave Blank**

Passed release pipeline.
Add release notes to release.
Cherry-pick release commit(s) as needed with merge request back to master branch
Celebrate and announce release

Candidate Release Notes

Please see our documentation page for more information on how to consume and deploy BigBang.

Upgrade Notices

Network polices have been added and are enabled by default for Authservice, ECK Operator, Gitlab, Monitoring, and SonarQube. Network policy enforcement for other packages will be added with future BigBang releases

# -- Global NetworkPolicies settings
networkPolicies:
  # -- Toggle all package NetworkPolicies, can disable specific packages with `package.networkPolicies.enabled`
  enabled: true

Packages

Pull in and replace chart then update for new release

Package	Type	Package Version	BB Version
Istio Operator	Core	`1.8.4`	`1.8.4-bb.1`
Istio Controlplane	Core	`1.8.4`	`1.8.4-bb.2`
Jaeger	Core	`1.22.0`	`2.19.1-bb.4`
Kiali	Core	`1.32.0`	`1.32.0-bb.1`
OPA Gatekeeper	Core	`3.3.0`	`3.3.0-bb.2`
Monitoring	Core	G: `7.1.3-1`, P: `2.22.0`, A: `0.21.0`	`11.0.0-bb.21`
ECK Operator	Core	`1.4.0`	`1.4.0-bb.2`
Elasticsearch Kibana	Core	E: `7.10.0` K: `7.10.2`	`0.1.11-bb.0`
Fluentbit	Core	`1.7.4`	`0.15.8-bb.1`
Cluster Auditor	Core	`0.3.2`	`0.2.0-bb.2`
Twistlock	Core	`21.04.412`	`0.0.4-bb.0`
Gitlab	Addon	`13.10.3`	`4.10.3-bb.6`
Gitlab Runners	Addon	`13.9.0`	`0.26.0-bb.1`
Mattermost Operator	Addon	`1.14.0`	`1.14.0-bb.1`
Mattermost	Addon	`5.34.2`	`0.1.5-bb.0`
MinIO Operator	Addon	`2.0.9`	`2.0.9-bb.3`
MinIO	Addon	`RELEASE.2020-11-19T23-48-16Z`	`2.0.9-bb.9`
Authservice	Addon	`0.4.0`	`0.4.0-bb.5`
Anchore	Addon	ENG: `0.9.3`, ENT: `3.0.2`	`1.12.13-bb.0`
SonarQube	Addon	`8.7.1` (w/ p1 plugins)	`9.2.6-bb.11`
Argocd	Addon	`1.8.4` (w/ p1 plugins)	`2.14.7-bb.5`
Velero	Addon	`1.5.3`	`2.14.8-bb.0`
Keycloak	Addon	`13.0.0`	`11.0.0-bb.0`
Nexus	Addon	`3.29.0-02`	`29.1.0-bb.0`

Changes in v1.9.0

!445: Nexus added to BB
!488: Authservice support external redis service
!499: Mattermost Operator optional network policies
!503: Sonarqube optional network policies
!504: Gitlab optional network policies
!510: ECK Operator optional network policies
!511: Authservice optional network policies
!513: Monitoring optional network policies
!514: Cluster Auditor & OPA Gatekeeper constraint-templates and added conditional enforcement

Known Issues

If the following error is seen on any helm releases scheme "" not supported try updating flux to latest ib images. A simple way to do this is by adding registry credentials to the flux-system namespace and applying the flux.yaml:

kubectl create secret docker-registry private-registry --docker-server=registry1.dso.mil --docker-username=<Your IronBank Username> --docker-password=<Your IronBank Personal Access Token> --docker-email=<Your E-mail Address> -n flux-system
curl https://repo1.dso.mil/platform-one/big-bang/bigbang/-/raw/master/scripts/deploy/flux.yaml | kubectl apply -f -

#329: Velero unable to restore PersistentVolumes.
#480: Jaeger fails to install on certain newer versions of kubernetes

Helpful Links

As always, we welcome and appreciate feedback from our community of users. Please feel free to:

Open issues here
Join our chat
Check out the documentation for guidance on how to get started

Future

Don't see your feature and/or bug fix? Check out our roadmap for estimates on when you can expect things to drop, and as always, feel free to comment or create issues if you have questions, comments, or concerns.

Edited Jun 07, 2021 by evan.rush

Admin message