P1BIGROCKS-2215

This EPIC involves enhancing the testing of merge requests (MR) to the MASTER branch of the big bang "umbrella" Helm chart.   A merge to MASTER is a candidate for release.   Therefore, these merge request should execute a broader, more comprehensive suite of tests to ensure that the candidate MR is of sufficient quality to warrant promotion to the mater branch.

The initial enhancement push is focused on creation and execution of tests that mimic the manual testing steps currently required during release testing.   

The test development can be implemented in several phases:

Phase 1:  Confirm UI is up and running for each Application

This phase is already being completed by EPIC &116 (closed)

Phase 2: UI Login Via SSO

Spike to determine how to test sso integration in the pipelines

This phase is focused on enabling the above UI tests to login to the UI using keycloak SSO. Investigation and Spike may be required to determine what's needed to support this effort.

Phase 3: Enhance Application Tests

Spike to determine which packages still need helm tests enhanced

This phase fill out more robust functional testing for each of the following components:   This could be completed with cypress or API tests.   Ideally the use of Helm Chart tests infrastructure can be added to each package to implement the test execution and the CI pipeline for bigbang extended to support execution of helm chart tests at merges.

Logging

Verify that Kibana is actively indexing logging information from components.

Cluster Auditor

Verify that the violations index is present and contains information

Monitoring

Verify that the Kubernetes Dashboards and metrics are present Verify that the istio dashboards are present

Prometheus

Verify that all apps are being scraped, no errors GitLab & Runners Create new public group with release name. Example release-1-8-0 Create new public project with release name. Example release-1-8-0

Anchore

Verify that this application can scan an image

Argocd

login with admin password Create application

*click* create application
application name: argocd-test
Project: default
Sync Policy: Automatic
Sync Policy: check both boxes
Sync Options: check both boxes
Repository URL: https://github.com/argoproj/argocd-example-apps
Revision: HEAD
Path: helm-guestbook
Cluster URL: https://kubernetes.default.svc
Namespace: argocd-test
*click* Create (top of page)

Delete application

Minio (Note that a helm chart test exists for this work)

Create bucket Store file to bucket Download file from bucket Delete bucket and files

Mattermost

Need to define a good test here

Velero

Backup PVCs and verify that a Restoration can be performed