Minio value merging doesn't work the way users expect
Bug
Description
The minio tenant pools are defined using a YAML array. This means that merging overriden values does not work. Consider the following:
- In minio's
chart/values.yamlwe define some basic rules for a single minio tenant pool - In a user's overrides, they may attempt to modify the minio tenant pool by specifying an array with a single entry with the same name, only providing the values that need to change
- In the bigbang umbrella chart
chart/templates/minio/helmrelease.yamlwe tell the helm chart to get its values from 3 separate places (the defaults, the overrides, etc), and merge them together
Unfortunately, when helm merges the yaml together, arrays are not merged - they are replaced by the last array in the merge order. This means that the minio tenant definition in the user's override completely replaces the default values provided in the chart, and is lacking many important configuration keys. This results in issues like this:
This is a known behavior, and is the appropriate documented behavior for the code being used here. There is no vendor-provided fix for the behavior, and we are unlikely to get one any time soon.
If you'd like to keep this issue open, the maintainers will be coming back from vacations soon-ish, so they can weigh in.
July 20, 2021
However it is confounding to the user and there is no indication that this has occurred except for the above failures - which are further confusing because it appears you are providing the values it asks for. There is no vendor-provided fix for this behavior.
The only way around this is to duplicate 100% of the values that were previously provided in the chart defaults, to ensure your tenant definition is complete. For example you must re-specify the securityContext, containerSecurityContext, and automountServiceAccountToken settings - just to name a few.
At the very least we should update the documentation in this regard. If we could convince the upstream minio maintainers to modify the chart to accept tenant pool definitions from a dictionary instead of an array, that would be best. Failing that, if we decided we wanted to maintain the code for it, we could try writing a helper function that does the merging intelligently in the helm chart itself.