Customer Feedback on Releases
Customers are asking for an update to the product release cadence and clarity on our semantic versioning.
TL;DR:
- Customers want to know what our SemVer means and have it documented.
- Releases are hard for our customers to keep up with, especially on high-side. Either we should consider investing in providing upgrade pathways for customers similar to this or we consider adjusting what goes into a release and how often.
Standardizing on a versioning scheme
- What does Major, Minor, Patch mean?
- What changes can customers expect in each?
- Can a breaking change come in a major release? Minor release? Patch?
- Will changes in minor/patch be backwards compatible?
- Major version changes indicate to a user that there are more things to look out for in the release notes.
- There is a need to clarify what these things means today.
Consider adjusting release schedule
-
Security Release separate from Feature Release.
- Preference for a security release would be bi-weekly, once a month for new features.
- On high-side and low-side, there are a lot of changes to reconcile in a short amount of time.
- Lots of default values to change per release, then deploy and validate per environment.
- Ex. changing default min/max cpu requirements across charts and workloads for each release. - A recent mandate for resource allocation to Big Bang impacted customers' resource requirements. This increased running costs.
-
Some customers maintain a lot of clusters, all of which need to be upgraded and tested every 2 weeks. This added complexity makes the upgrade process harder with a limited schedule. Integration testing of mission capabilities on the high-side takes time and there isn't much time available in SCIFs sometimes - requiring weekend work to verify both platform and apps work.
-
Consider adding an LTS branch
- This would introduce less breaking changes to the product.
- LTS would get critical security updates, minor updates (configs, define these), and anything else that will not strictly break anything.
- Continue using release branches but potentially less of them.
Adding Upgrade Paths
- It is easy to fall behind the release cadence, especially on the high-side.
- The current (n-2) upgrade path creates more work for customers to verify Big Bang and workload functionality.
- Consider easing this, or offering an upgrade path for some releases
General Release-related notes
- Big Bang should define what "Feature Complete" means.
- How long do we continue our 2-week cycles?
- Big Bang should provide additional transparency on what is driving the release cadence & the requirements.
- ex. Who decides what features get added, cut, or improved upon in each release? How can this process be made more accessible?