Upgrade elasticsearch-kibana package to 7.10.X versions

Elasticsearch version 7.10.0 is available in IronBank Kibana version 7.10.2 (confirmed compatibility in elastic docs) is available in IronBank.

Upgrade image versions from current 7.9.2 in package chart and confirm compatibility and upgrade success from previous version.

added to epic &29 (closed)

added kindenhancement logging + 1 deleted label

assigned to @ryan.j.garcia

removed from epic &29 (closed)

Looks like Rolling Upgrade can be performed https://www.elastic.co/guide/en/elasticsearch/reference/7.11/rolling-upgrades.html#_upgrading_your_cluster

Will need to run the following commands PRE-UPGRADE:

curl -XPUT -u "elastic:PaSsWoRd" "localhost:9200/_cluster/settings?pretty" -H 'Content-Type: application/json' d' { "persistent": { "cluster.routing.allocation.enable": "primaries" } }'

curl -XPOST -u "elastic:PaSsWoRd" "localhost:9200/_flush/synced?pretty"

And post-upgrade:

curl -XPUT -u "elastic:PaSsWoRd" "localhost:9200/_cluster/settings?pretty" -H 'Content-Type: application/json' -d' { "persistent": { "cluster.routing.allocation.enable": null } }'

Testing using kubernetes job resource that can be run as a helm hook, check for a combination of a previous chart version and new ES value in the HelmRelease object properties using the JSON output or curling the k8s API:

curl https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT/api/v1/namespaces/bigbang/helmreleases/ek -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" | grep lastAppliedRevision

kubectl get hr ek -n bigbang -o jsonpath='{.status.lastAppliedRevision}'

kubectl get hr ek -n bigbang -o jsonpath='{.spec.values.elasticsearch.version}'

added statusdoing label and removed 1 deleted label

added statusblocked label and removed statusdoing label

Adding blocked label. Need to find good way to handle rolling upgrades between chart versions so customers and CI aren't screwed when doing a version bump and shard rerouting and writing is adversely affected.

changed iteration to Big Bang Iterations Mar 9, 2021 - Mar 15, 2021

removed milestone %1.3.0

Successfully able to upgrade 7.9.X cluster to 7.10.X without interaction using: https://repo1.dso.mil/platform-one/big-bang/apps/core/elasticsearch-kibana/-/blob/bb-224/chart/templates/bigbang/pre-upgrade-job.yaml https://repo1.dso.mil/platform-one/big-bang/apps/core/elasticsearch-kibana/-/blob/bb-224/chart/templates/bigbang/post-upgrade-job.yaml

Cleaning up and creating MR for review and input.

added statusdoing label and removed statusblocked label

Opened MR in elasticsearch-kibana package https://repo1.dso.mil/platform-one/big-bang/apps/core/elasticsearch-kibana/-/merge_requests/26

created merge request !342 (merged) to address this issue

mentioned in merge request !342 (merged)

added to epic &63 (closed)

added statusblocked label and removed statusdoing label

removed statusblocked label

added statusdoing label

set weight to 4

changed iteration to Big Bang Iterations Apr 6, 2021 - Apr 20, 2021

assigned to @micah.nagel

changed iteration to Big Bang Iterations Apr 21, 2021 - May 4, 2021

removed weight

set weight to 2

changed milestone to %1.8.0

Pushing to release 1.8.0.

Need to up the default flux timeout, plus set ~25m timeout for the EK HelmRelease resource specifically in 1.7.0 to allow for smoother upgrade.

marked this issue as related to #388 (closed)

added statusblocked label and removed statusdoing label

changed iteration to Big Bang Iterations May 5, 2021 - May 18, 2021

added statusdoing label and removed statusblocked label

added statusreview label and removed statusdoing label

changed iteration to Big Bang Iterations May 19, 2021 - Jun 1, 2021

closed with merge request !342 (merged)

mentioned in commit 95b3ad6d

removed statusreview label

mentioned in merge request big-bang/product/packages/policy!235 (closed)

mentioned in merge request big-bang/product/packages/policy!241 (closed)

mentioned in merge request big-bang/product/packages/policy!240 (closed)

mentioned in merge request big-bang/product/packages/policy!242 (closed)

mentioned in merge request big-bang/product/packages/policy!243 (closed)

mentioned in merge request !5545 (merged)