mimir update to 5.5.1-bb.6

Package Merge Request

Package Changes

https://repo1.dso.mil/big-bang/product/packages/mimir/-/blob/5.5.1-bb.6/CHANGELOG.md

Package MR

big-bang/product/packages/mimir!49 (merged) and big-bang/product/packages/mimir!54 (merged)

For Issue

Closes big-bang/product/packages/mimir#62 (closed) and big-bang/product/packages/mimir#65 (closed)

Upgrade Notices

• mimir-distributed has deprecated the nginx service and related values in favor of gateway.
  • All remote_write scraper configurations not included in Big Bang should be updated to point to mimir-gateway instead of nginx.

Important- The upgrade steps below are applicable only when MinIO is enabled for Mimir. As a reminder, MinIO is only supported for development environments. If your environment is not using Big Bang's MinIO for Mimir, you can ignore the following upgrade notice. If Big Bang's MinIO is enabled for Mimir (minio-tenant.enabled to true), remember to test upgrades and maintain proper backups.

Upgrading from 5.5.1-bb.4 involves changes to Tenant immutable server field that requires 1) the mimir-mimir-minio-tenant tenant to be deleted, 2) remove any finalizers protecting the persistent volumes (pv) that were used by the mimir-mimir-minio-tenant tenant, and 3) delete the persistent volumes and persistent volume claims (pvc) used by the mimir-mimir-minio-tenant tenant to free up the persistent volumes for the new MinIO tenant before upgrading the release. Due to the recreation of the tenant, any historical metric data in the MinIO tenant buckets, pre-upgrade, will be lost as part of the tenant redeployment.

The Mimir upgradeJob.enabled field in values.yaml can deploy a pre-upgrade job that automates the required tenant, pv, and pvc deletion without any additional steps by using a helm pre-upgrade hook, so no additional actions is required. It should be noted, 1) the buckets containing historical metrics will be replaced so historical metrics collected will no longer available and 2) a brief outage is expected during upgrade while the mimir-mimir-minio-tenant tenant is being rolled out.

However, if you would rather manually complete the pre-upgrade steps listed above prior to upgrade, then you would need to set the upgradeJob.enabled=false in the values.yaml and follow steps outlined below. The below command assumes that the Mimir package is deployed in the default Big Bang mimir namespace for Mimir, one should look to confirm the namespace of their Mimir deployment:

1. Deleting the mimir-mimir-minio-tenant Tenant

kubectl delete tenant mimir-mimir-minio-tenant -n mimir

2. Remove any Finalizers for the Persistent Volumes that was used by the Tenant

kubectl patch pv <name of the pv used by the Minio Tenant> --type=json -p '[{"op": "remove", "path": "/metadata/finalizers"}]'

3. Delete the Persistent Volumes that was used by the Tenant

kubectl delete pv <name of the pv used by the Minio Tenant>

4. Delete the Persistent Volumes Claims that was used by the Tenant

kubectl delete pvc <name of the pvc used by the Minio Tenant> -n mimir

Once the resources have been deleted, you can upgrade the release

chart/templates/grafana/values.yaml

@@ -109,7 +109,7 @@ datasources:
       type: prometheus
       uid: mimir
       access: proxy
-      url: http://mimir-mimir-nginx.mimir.svc.cluster.local/prometheus
+      url: http://mimir-mimir-gateway.mimir.svc.cluster.local/prometheus
       editable: true
     {{- end }}
     {{- if .Values.monitoring.enabled }}