Regression Testing Myths

Regression Testing is a very commonly misunderstood term and has been made synonymous with retest using automation. I have written about this before, but I feel compelled to write once more about it so that there is clarity in the community about what regression testing really is. Let’s bust the regression testing myths!

What is Regression

Regression is a degrade of quality from a known standard. You had a software that was working fine, but because of changes or new features that you have introduced or because of the other dependencies that were introduced/updated, the software does not work with same quality anymore and has degraded. Note that if it is an upgrade of functionality, I wouldn’t be concerned, but since there is a degrade, I need to look at what happened.

What is Regression Testing

There are two parts to regression testing.

• The tests that are conducted to figure out what went wrong
• The tests that are conducted after a fix has been applied to the regression

Note that I am using the phrase ‘tests that are conducted’, and I have not mentioned automation. Regression tests need not be automation. Automation is only an aid for the tests to be conducted and it is not that by default regression tests are automated. Yes, because of time criticality and to improve efficiency and effectiveness, you can decide to introduce automation, but regression tests are not by default automated.

What about ‘Manual Regression Tests’?

Tests are by default done by human with the aid of automation. When regression tests are done by human, it cannot be called by ‘manual’, because at some point of time or other, humans will be involved in looking at the tests to be done, planning those tests, invoking automation as and when required to aid in tests. It is a sad story that regression tests are being equated to ‘automation suites’ to be executed. Some even talk about ‘selecting the automated tests instead of blindly executing the suite’! This blind runaway approach to stick to automation without due diligence of what the tests do, whether they are sufficient and necessary, and and what point human oversight is needed leads to customer place outages.

Regression vs Retest

When you have a new release and you want to invoke some automation to test whether existing features are working properly, it is called ‘Retest’, not ‘Regression’. Regression test associated only when there is degrade of existing functionality. Period.

Conclusion

Using CI/CD is great. Invoking automation for repetitive steps in testing (note: NOT ‘test automation’!) is great, but equating invoking an automation suite for regression testing or naming a retest as regression are not! Let’s bust the regression testing myths. We need to be very clear about the scope and definition of regression testing such that we provide superior quality to the customers. If your organisation needs help in planning or strategizing regression testing in your projects, feel free to get in touch with me. Glad to help!

The post Regression Testing Myths appeared first on Venkat Ramakrishnan.