`mimir` blocks BB upgrade in all-packges pipeline

Issue

When an all-packages pipeline runs in BB umbrella, mimir does not upgrade cleanly.

Example MR with failing upgrade pipeline: big-bang/bigbang!5723

Relevant logs:

timed out waiting for the condition on jobs/mimir-mimir-smoke-test

Acceptance Criteria

• No mimir components block the BB upgrade in all-packages pipelines

marked this issue as related to #60 (closed)

added kindci mimir priority5 statusready-to-work teamObservability labels

set weight to 2

mentioned in merge request big-bang/bigbang!5718 (merged)

marked this issue as blocking kyverno-policies#150 (closed)

changed iteration to Big Bang Iterations Feb 4, 2025 - Feb 17, 2025

assigned to @bjacksonfv

added statusdoing label and removed statusready-to-work label

Interestingly; it passes locally on k3d

➜  bigbang git:(master) helm test mimir-mimir -n bigbang --timeout 10m
NAME: mimir-mimir
LAST DEPLOYED: Wed Feb 12 15:43:16 2025
NAMESPACE: mimir
STATUS: deployed
REVISION: 1
TEST SUITE:     mimir-mimir-smoke-test
Last Started:   Wed Feb 12 10:53:27 2025
Last Completed: Wed Feb 12 10:53:39 2025
Phase:          Succeeded

seeing the same here, working on #60 (closed)

I do get ~2 restarts on the smoke-test container/pod, but always stays within the limit (5)

removed iteration Big Bang Iterations Feb 4, 2025 - Feb 17, 2025

changed iteration to Big Bang Iterations Feb 18, 2025 - Mar 3, 2025

mentioned in merge request big-bang/bigbang!5793 (merged)

I'm fairly confident the underlying issue has been identified and resolved in big-bang/bigbang!5793 (merged)

I will try to get MR merged if I can get a successful clean_install with all-packages pipeline with the Mimir smoke tests passing (It may still fail on other packages)

Pipelines keep failing before the mimir-smoke-test even runs; but there was at least one job that ran them, and the smoke test succeeded https://repo1.dso.mil/big-bang/bigbang/-/jobs/42611867

Going to consider this done

removed iteration Big Bang Iterations Feb 18, 2025 - Mar 3, 2025

changed iteration to Big Bang Iterations Mar 4, 2025 - Mar 17, 2025

closed

removed statusdoing label

mentioned in issue #60 (closed)