Resolve "Helm test failure on upgrade job aborts the upgrade testing"

Closes #824 (closed)

changed milestone to %1.20.0

added kindbug teamci/pipelines labels

changed the description

added 1 commit

• 890ed309 - comment exit in .test_bigbang to test

Compare with previous version

It looks like the "exit" in https://repo1.dso.mil/platform-one/big-bang/bigbang/-/blob/master/.gitlab-ci.yml#L106 is both exiting the function, and the job, rather than just the function. Will try having it pass the "123" error code rather than exiting directly on it.

added 1 commit

• e58488d7 - added if statement to handle exits

Compare with previous version

https://repo1.dso.mil/platform-one/big-bang/bigbang/-/jobs/7428997

I think this job shows the correct output that we're looking for. (Kiali fails, but the error code is recorded and the job exits with correct 123 error code).

Working to improve logging (adding warnings when a helm test fails) and cleaning up my changes.

added 1 commit

• a2b354b8 - improved logging and error handling

Compare with previous version

added 1 commit

• 5597b6d4 - fix font, added 123 exit for upgrade

Compare with previous version

Most recent pipeline shows correct behavior. Modified how error handling is done so that if jobs fail with anything other than 123 the pipeline exits. Improved logging to make it more clear what's happening why it's passing with errors.

marked this merge request as ready

added statusreview label

requested review from @BrandenCobb and @micah.nagel

LGTM.

approved this merge request

Otherwise LGTM!

resolved all threads

added 2 commits

• 5a8e6c98 - remove intitializing exit_code
• 66cb5530 - refined how EXIT_CODE is handled

Compare with previous version

added 1 commit

• 38cf9e5c - condensed if statements

Compare with previous version

added 1 commit

• f0b76b31 - fixed variable name and added to rke2 test

Compare with previous version

added 1 commit

• 6359f884 - changed && to if statements

Compare with previous version

Made some changes to improve exit flag logic. Exit flag is now either undefined or set, and each function that calls test_bigbang exits based off of that. Latest pipeline indicates that it's working correctly.

approved this merge request

LGTM.

approved this merge request

merged

mentioned in commit d63f088f