Continuous testing or continually testing?

One of the first things I did when joining my previous organisation was pitch testing throughout the SDLC. I wanted to share the importance of testing everything that we do.

However not everyone shares that idea of continuous testing throughout the SDLC. For some continuous testing is continuously running the same tests over and over. If you don’t have them fully automated E2E, repeat them manually over and over.

Skipping over how expensive it is to focus on E2E tests, this really misses the entire point of continuous testing. Instead of continuously trying to discover more about our application and challenging everything that we are doing, all we are doing is repeating that same check… and getting the same answers.

Whilst there can definitely be value in repeating tests when you have fear of a risk of regressions (and automation makes that so much easier), the true value in testing is to be trying new things, new ideas and devising new tests.

This is partly why I think people wedded to writing traditional manual test cases are doomed to struggle with test cases. It is a slow, tedious and usually fruitless endeavour.

Testing is more than checking. We need to be continuously experimenting, learning and discovering behaviours. When looking to regression test, don’t ask “what test cases do we have?” but instead “what are the risks and concerns that I have?” Ask “what information do we need to uncover?”. Then devise tests, ideally exploratory or at least looser than rigid test cases in order to uncover the information that you need and the risks associated with that.

In a future post I intend to give a case study or two on some interesting approaches that I’ve looked at.

I originally meant to post something around this about 2 years ago – better late than never! What I won’t state is whether it was still an issue at time of writing…