Release 1.11.0
Incomplete Draft
Release Process
0. Copy this checklist to release issue/ticket
1. Release Prep
- Verify that the previous release branch commit hash matches the last release tag. Investigate with previous RE if they do not match
-
Create release branch with name. Ex:
release-1.9.x - Build draft release notes, see release_notes_template.md
-
Release specific code changes. Make the following changes in a single commit so it can be cherry picked into master later.
-
Bump self-reference version in
base/gitrepository.yaml -
Update chart release version
chart/Chart.yaml -
Bump badge at the top of
README.md -
Update
/Packages.mdwith any new Packages -
Update CHANGELOG.md with links to MRs and any upgrade notices/known issues. release-diff update link for release
-
Update README.md using
helm-docs. Overwrite the existing readme file.# from root dir of your release branch docker run -v "$(pwd):/helm-docs" -u $(id -u) jnorwood/helm-docs:v1.5.0 -s file -t .gitlab-ci/README.md.gotmpl --dry-run > README.md
-
2. Test and Validate Release Candidate
Deploy release branch on Dogfood cluster
- Connect to Cluster
-
Update
bigbang/base/kustomization.yaml&bigbang/prod/kustomization.yamlwith release branch. -
Verify cluster has updated to the new release
-
Packages have fetched the new revision and match the new release
-
Packages have reconciled
# check release watch kubectl get gitrepositories,kustomizations,hr,po -A # if flux has not updated after 10 minutes. flux reconcile hr -n bigbang bigbang --with-source # if it is still not updating, delete the flux source controller kubectl get all -n flux-system kubectl delete pod/source-controller-xxxxxxxx-xxxxx -n flux-system
-
Confirm app UIs are loading
- anchore
- argocd
- gitlab
- tracing
- kiali
- kibana
- mattermost
- minio
- alertmanager
- grafana
- prometheus
- sonarqube
- twistlock
- nexus
- TLS/SSL certs are valid
Logging
- Login to kibana with SSO
- Kibana is actively indexing/logging.
Cluster Auditor
- Login to kibana with SSO
- violations index is present and contains images that aren't from registry1
Monitoring
- Login to grafana with SSO
- Contains Kubernetes Dashboards and metrics
- contains istio dashboards
- Login to prometheus
- All apps are being scraped, no errors
Kiali
- Login to kiali with SSO
Sonarqube
- Login to sonarqube with SSO
GitLab & Runners
-
Login to gitlab with SSO
-
Create new public group with release name. Example
release-1-8-0 -
Create new public project with release name. Example
release-1-8-0 -
git clone and git push to new project
-
docker push and docker pull image to registry
docker pull alpine docker tag alpine registry.dogfood.bigbang.dev/GROUPNAMEHERE/PROJECTNAMEHERE/alpine:latest docker login registry.dogfood.bigbang.dev docker push registry.dogfood.bigbang.dev/GROUPNAMEHERE/PROJECTNAMEHERE/alpine:latest -
Edit profile and change user avatar
-
Test simple CI pipeline. sample_ci.yaml
Anchore
- Login to anchore with SSO
-
Scan image in dogfood registry,
registry.dogfood.bigbang.dev/GROUPNAMEHERE/PROJECTNAMEHERE/alpine:latest
Argocd
- Login to argocd with SSO
-
Logout and login with
admin. password reset -
Create application
*click* create application application name: argocd-test Project: default Sync Policy: Automatic Sync Policy: check both boxes Sync Options: check both boxes Repository URL: https://github.com/argoproj/argocd-example-apps Revision: HEAD Path: helm-guestbook Cluster URL: https://kubernetes.default.svc Namespace: argocd-test *click* Create (top of page) - Delete application
Minio
- Create bucket
- Store file to bucket
- Download file from bucket
- Delete bucket and files
Mattermost
- Login to mattermost with SSO
- Elastic integration
Velero
-
Backup PVCs velero_test.yaml
kubectl apply -f ./velero_test.yaml # exec into velero_test container cat /mnt/velero-test/test.log # take note of log entries and exit execvelero backup create velero-test-backup-1-8-0 -l app=velero-test velero backup get kubectl delete -f ./velero_test.yaml kubectl get pv | grep velero-test kubectl delete pv INSERT-PV-ID -
Restore PVCs
velero restore create velero-test-restore-1-8-0 --from-backup velero-test-backup-1-8-0 # exec into velero_test container cat /mnt/velero-test/test.log # old log entires and new should be in log if backup was done correctly
Keycloak
3. Create Release
-
Create release candidate tag based on release branch. Tag EX:
1.8.0-rc.0.Message: release candidate Release Notes: **Leave Blank** - Passed tag pipeline.
-
Create release tag based on release branch. Tag EX:
1.8.0.Message: release 1.x.x Release Notes: **Leave Blank** - Passed release pipeline.
- Add release notes to release.
- Cherry-pick release commit(s) as needed with merge request back to master branch
- Celebrate and announce release