As previously discussed with Marty, any further flakey tests (timing issues) with Cypress will be skipped for now but not removed.
- Timing issues are effecting Auto Screens UI & PostgreSql tests
- We will receive coverage of these tests via QA Wolf
- The tests are not removed so we still have reference for generating relative API tests
A few changes to maintain what we have for the nightly smoke build
autoScreensUI
- Adding a 1 second wait. This seems essential now for the smoke run on CI. Was difficult to reduce the error locally but definitely timing related
userManagement
- Reworking how we confirm a user has no applications
- The text has also changed - updated it too
Skipping tests for Oracle & datasourceWizard
Both are covered during general regressions
There are timing issues associated with the CI Smoke run
This will help reduce the smoke build run time
The oracle test file was only covering a basic check of invalid configuration (Not essential for the smoke run)
rest.spec.js
- Test was flakey on last nights smoke run
- Updated to specifically search for query-buttons and then add query
commands.js
- Increasing timeout values for createTable & createDatasourceScreen commands
createComponents
- Updated how the test gets nav-items from the side nav bar
commands
- Updating maximum timeout value for cy.visit instances (this seems specific to the CI runs)
- Updated createUser command to ensure better timing between the create user modals
Update createTable test command
createTable command
- Changing how a check is completed when dealing with the create table modals
appPublishWorkflow
- Increasing timeout value related to the app table status
- This increased timeout value is needed specifically for CI runs
MySQL
- Timing issue for the smoke build
- Including an intercept after saving a query and then confirming it exists. Sometimes Cypress tries to find the query within the nav item section before the query has finished
PostgreSQL
- Timing issue for the smoke build
- Sometimes Cypress will continue the test before the page has finished reloading. Increased timeout value here to ensure the page loads. Also ensuring the entire nav item section is visible
In some cases Cypress is running too fast, meaning that the test runs ahead of what is actual seen on Budibase.
This issue mainly occurs at the start of the smoke build and around logging in.
I am attempting to combat this by ensuring the url includes 'apps' after logging in
We don't need a specific test file for this.
The test file will remain for now but will not run. The test will be reworked into a different test file or as part of the API testing
Some test changes for test files within the adminAndManagement folder
- Changes related to UI
- Restructured accountPortals tests
- Changes related to changing passwords (flow is different now)
UI changes for users requires some test updates
- User commands updates
- New user command - setUserRole
- Test updates to adminAndManagement + Datasource tests
- Some test skips as they need reworked (specifically user management tests)
PostgreSql
- Added intercept when saving a query, sometimes it runs too fast
createTable command
- Improved to ensure better timing, and that table is correctly created
Data-cy IDs are gone for components (I assume due to the UI redesign)
For now I have created a new command to search and add a component. Tests have been updated to reflect this
Updated the current createScreen test file:
- Reworked a test related to access levels
- Tests for screens of all access levels and deleting all screens (and starting new screen journey)
- Added commands to delete screen and delete all screens
This test spec file fires off against every PR. There is some timing issues currently.
I am making a few changes in relation to timeout values (a small restructure too) in hopes this helps with the PR builds.
Tests pass locally for me, so there may be more changes required against this spec file as I check its performance
Getting close.... but there are still some minor timing issues against the smoke build.
I am increasing a few timeout values within postgreSQL & commands files
It seems that at times, Cypress is running so fast that it does not type entire words (it sometimes misses the first couple of letters.
I've put a small change in the userManagement test file to try and revolve this.
- Adding a wait and clear in association with the type command
Still some timing issues against the smoke build, but less this time.
- Mainly adding timeouts against cy.visit
- Improving some structure with the close modal command in a few places
This time it is adding appropriate deleteAllApps commands and adding waits
- The waits seem to be essential for the smoke build.
- Tests pass locally, but timing issues causes failure on the smoke build
Changes have been made to the following test files to remove smoke build failures:
- addRadioButtons.spec.js
- accountPortals.spec.js
- userManagement.spec.js
- userSettings.spec.js
All tests pass locally, just having issues with timing on the CI run
Also reduced test retries (runs a total of 2 times instead of 3, the 3rd run does not add any value)
Pedro Silva
Mitch-Budibase
Andrew Kingston
mike12345567
Martin McKeaveney
deanhannigan