Update Argo Cypress test to work in BB CI
Argo tests currently will fail if enabled in BB CI. It appears to be a timing issue. See below logs.
====================================================================================================
(Run Starting)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Cypress: 5.0.0 │
│ Browser: Chrome 83 (headless) │
│ Specs: 1 found (argocd.spec.js) │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────
Running: argocd.spec.js (1 of 1)
Browserslist: caniuse-lite is outdated. Please run:
npx browserslist@latest --update-db
Basic ArgocD
1) Check ArgoCD is accessible
0 passing (32s)
1 failing
1) Basic ArgocD
Check ArgoCD is accessible:
CypressError: Timed out retrying: `cy.click()` failed because this element:
`<span class="applications-list__title">guestbook</span>`
is being covered by another element:
`<div class="sliding-panel__outside"></div>`
Fix this problem, or use {force: true} to disable error checking.
https://on.cypress.io/element-cannot-be-interacted-with
at $Cy.ensureDescendents (https://argocd.bigbang.dev/__cypress/runner/cypress_runner.js:160969:26)
at ensureDescendents (https://argocd.bigbang.dev/__cypress/runner/cypress_runner.js:149596:8)
at ensureDescendentsAndScroll (https://argocd.bigbang.dev/__cypress/runner/cypress_runner.js:149603:14)
at ensureElIsNotCovered (https://argocd.bigbang.dev/__cypress/runner/cypress_runner.js:149732:5)
at runAllChecks (https://argocd.bigbang.dev/__cypress/runner/cypress_runner.js:149869:52)
at retryActionability (https://argocd.bigbang.dev/__cypress/runner/cypress_runner.js:149884:16)
at tryCatcher (https://argocd.bigbang.dev/__cypress/runner/cypress_runner.js:9852:23)
at Function.Promise.attempt.Promise.try (https://argocd.bigbang.dev/__cypress/runner/cypress_runner.js:7126:29)
at tryFn (https://argocd.bigbang.dev/__cypress/runner/cypress_runner.js:164075:24)
at whenStable (https://argocd.bigbang.dev/__cypress/runner/cypress_runner.js:164116:12)
at https://argocd.bigbang.dev/__cypress/runner/cypress_runner.js:163634:16
at tryCatcher (https://argocd.bigbang.dev/__cypress/runner/cypress_runner.js:9852:23)
at Promise._settlePromiseFromHandler (https://argocd.bigbang.dev/__cypress/runner/cypress_runner.js:7787:31)
at Promise._settlePromise (https://argocd.bigbang.dev/__cypress/runner/cypress_runner.js:7844:18)
at Promise._settlePromise0 (https://argocd.bigbang.dev/__cypress/runner/cypress_runner.js:7889:10)
at Promise._settlePromises (https://argocd.bigbang.dev/__cypress/runner/cypress_runner.js:7969:18)
From Your Spec Code:
at Context.eval (https://argocd.bigbang.dev/__cypress/tests?p=cypress/integration/argocd.spec.js:133:30)
(Results)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 1 │
│ Passing: 0 │
│ Failing: 1 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 1 │
│ Video: true │
│ Duration: 32 seconds │
│ Spec Ran: argocd.spec.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
(Screenshots)
- /test/cypress/screenshots/argocd.spec.js/Basic ArgocD -- Check ArgoCD is accessi (1280x720)
ble (failed).png
(Video)
- Started processing: Compressing to 35 CRF
- Finished processing: /test/cypress/videos/argocd.spec.js.mp4 (1 second)
====================================================================================================
(Run Finished)
Spec Tests Passing Failing Pending Skipped
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✖ argocd.spec.js 00:32 1 - 1 - - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
✖ 1 of 1 failed (100%) 00:32 1 - 1 - - Activity
added argocd kindci teamci/pipelines labels
changed milestone to %1.19.0
changed iteration to Big Bang Iterations Oct 5, 2021 - Oct 18, 2021
assigned to @blloyd
added statusdoing label
changed milestone to %1.20.0
changed iteration to Big Bang Iterations Oct 19, 2021 - Nov 1, 2021
unassigned @blloyd
removed statusdoing label
assigned to @blloyd
added statusdoing label
created merge request !50 (merged) to address this issue
mentioned in merge request !50 (merged)
changed milestone to %1.21.0
changed iteration to Big Bang Iterations Nov 2, 2021 - Nov 15, 2021
I think some UI changes have been made that broke the Cypress tests. It does not appear to have been a timing issue, as the box that was in the way of the item it wanted to click on (the title of the guestbook app) does not go away until you click the X at the top right of that box, and I've added a line to do that.
There are also other small changes. For example, "Sync OK" is now shown as "Synced", and we can't do a cy.contains for anything that contains 'Healthy' because it will find another element that happens to also contain "Healthy". So I've changed it to make sure it checks specific elements for those keywords.
The test is currently failing because the test app never syncs, which is not a cypress/UI issue, but I will continue to work on it.
changed milestone to %1.22.0
changed iteration to Big Bang Iterations Nov 16, 2021 - Nov 29, 2021
-
The cypress test itself is working properly, but the challenge at the moment is that the guestbook app which is deployed as part of the test never becomes healthy, so it fails. I'm troubleshooting that at the moment, and in recent commits have added a function to take screenshots of the event logs from within the UI.
added statusreview label and removed statusdoing label
added statusdoing label and removed statusreview label
-
Tests are now working properly in package CI and bigbang CI (including while using the all-packages tag).
In addition to some UI changes that prevented the old version from working, there are also some quirks that need to be handled when the cluster is under high load, such as the necessity of closing and reopening a pane to see if certain text shows up, clicking on a synchronize button twice and then seeing if it begins synchronizing and clicking on it again if it's not, etc.
Other changes include:
- Upgraded to Gluon 0.2.4
- Now writing the progress of the test to the container's stdout (which get included in the CI logs), so it's easy to see how many times a loop needed to run, how far the test got, etc.
- Added a function to go into the test application's events and take screenshots at an arbitrary interval, this is not being called now but can be uncommented when debugging.
added statusreview label and removed statusdoing label
closed with merge request big-bang/bigbang!1072 (merged)
removed statusreview label
