“one tiny change, one massive issue!”
When planning a continuous integration (CI) strategy, it’s important to ensure that code is deployed and tested appropriately for every release.
Most CI suites will test unit tests for every build, however even with Test Driven Development (TDD), unit tests are usually functional, they don’t test interactions, the flow between pages. Companies aim for 100% coverage but rarely achieve it.
Small changes can easily introduce breaks in flow, but these breaks may not be picked up by normal testing. Its important that your CI strategy includes a test progress/team who can focus on front-end, user oriented end-to-end tests in addition to specific functional tests, using automated tools.
Situations can occur where code works fine for an anonymous user, but logged in users who may have a different payment flow (or interface) could encountered an error. An issue as simple as an additional include causing a Javascript conflict in a single page or flow can cause an error which would otherwise go un-noticed preventing your users from having a successful experience.
Javascript errors are client based, so your web server logs wont pickup these as you might a server side error.
Selenium is the most widely used browser test tool today. We usually write selenium scripts, which can be initially recorded with assertions to inject and validate data during a user flow. Capturing screen shots and detecting failures returning an state (pass/fail).
These tests coupled with services such as Browserstack (or Appium), can confirm that a user can reach every section that a user should be able to reach on an array of browsers and/or devices.
Regular reviews to ensure that these flows are properly mapped out, and maintained are essential to minimising regression faults before they go into production.