Exploratory Testing vs. Scripted Testing

There are two different approaches to software testing — Scripted Testing and Exploratory Testing.

Scripted Testing is based on the sequential examination of requirements, followed by test case design and documentation, followed by test case execution.
The scripted tester’s slogan is, “Plan your work, work your plan.”

On the other hand, Exploratory Testing is a very different approach.
Instead of a sequential approach, it emphasizes concurrent product learning, test design, and test execution.
In this approach, testers design and execute tests while exploring the product.

Planning is described as “figuring out what to do next.”
While planning is crucial for effectiveness and efficiency, it’s still unsettled when and how to plan.

Scripted Testing suggests an idea of early test design to find defects in requirements and design before the code is written and the system put into production.

Exploratory Testing challenges the idea of designing tests so very early in a project when our knowledge is still developing. Its focus is on continuous learning and adaptability.

Let’s take a closer look at each of them.

Scripted Testing

In Scripted Testing approach testers follow the script which includes well-documented test cases along with instructions and expected outcomes.

There is very minimal to no deviation from the script.

It ensures that every functionality must be tested and tests are to be passed.

Definition

ISTQB glossary provides the following meaning for Scripted Testing:

Testing (manual or automated) that follows a test script.

When to use Scripted Testing?

Scripted testing is well-suited for testing high-risk applications (e.g. financial applications). This approach is well-suited when there is enough time for documentation and planning. Scripted testing requires skilled resources and detailed coverage and verification.

Advantages of Scripted Testing

1. Plays a role of formal documentation: in situations where a good requirements specification is lacking, test cases, become the de facto requirements specification, including the results that demonstrate which requirements were fulfilled and which were not.
2. Coverage: because scripted tests are created from requirements, design, and code, all important attributes of the system will be covered by tests and this coverage can be defined and measured.
3. Well-suited for automation: because the tests are defined in detail, they are more easily automated.
4. Scripted tests, when written to the appropriate level of detail, can be run by people who would otherwise not be able to test the system because of a lack of domain knowledge or lack of testing knowledge.
5. By creating the tests early, we can discover what we don’t know and focus on the “big picture.”
6. There might be specific tests that must be executed the same way, to act as a benchmark.
7. Since tests are created early in the development process, this may afford extra time during the important test execution stage.

Disadvantages of Scripted Testing

1. Scripted testing is very dependent on the requirements’ quality. Will the requirements really be complete, consistent, unambiguous, and stable enough as the foundation for scripted testing? Perhaps not.
2. Scripted testing is limited because it follows the script. Many interesting defects could be missed with this approach.
3. Scripted testing can be more tedious and time-consuming.

Exploratory Testing

Rather than a sequential examination of requirements, followed by test case design and documentation, followed by test case execution, Exploratory Testing is “simultaneous learning, test design, and test execution.”

The tester designs and executes tests while exploring the product.

In Exploratory Testing, testers are involved in minimum planning and maximum test execution.

Test design and test execution activities are performed in parallel typically without formally documenting the test conditions, test cases, or test scripts.

At the same time, notes will be taken during the Exploratory Testing session (what is tested, any observations or defects found, and thoughts about possible further testing), so that a report can be produced afterward.

A key aspect of Exploratory Testing is learning: learning by the tester about the software, its use, its strengths, and its weaknesses. Following its name, Exploratory Testing is about exploring, and finding out about the software, what it does, what it doesn’t do, what works, and what doesn’t work. The tester is constantly making decisions about what to test next and where to spend time.

Definition

ISTQB glossary provides the following meaning for Exploratory Testing:

An approach to testing in which the testers dynamically design and execute tests based on their knowledge, exploration of the test item and the results of previous tests.

When to use Exploratory Testing?

This is an approach that is most useful when there are no or poor specifications and when time is severely limited.

Exploratory Testing is also vital whenever choosing the next test case to be run cannot be determined in advance, but should be chosen based on previous tests and their results.

It can also serve to supplement other, more formal testing, helping to establish a higher level of confidence in the software, and ensuring that the most serious defects have been found.

Advantages of Exploratory Testing

1. Exploratory Testing is useful when you are asked to provide rapid feedback on a product’s quality on short notice, with little time, off the top of your head, when requirements are vague or even nonexistent, or early in the development process when the system may be unstable.
2. Exploratory Testing is more flexible, creative, and efficient.
3. Exploratory Testing is useful when, once a defect is found, we want to explore the size, scope, and variations of that defect to provide better feedback to developers.

Disadvantages of Exploratory Testing

1. We remember that for Scripted Testing test cases design begins during the requirements gathering and design phases, and so defects can be identified and corrected earlier. On the other hand, Exploratory Testing has no such ability to prevent defects.
2. There is no documentation for reference.
3. Heavily relies on the tester’s knowledge, experience, and skillset.

Factors that affect Exploratory Testing

• what to test
• what documents (requirements, design, etc.) are available
• what tactics to use
• what kinds of defects to look for
• what risks are involved
• testing purpose for a particular phase
• available time
• testers' experience and skills
• previous test results

Conclusions

Both Exploratory Testing and Scripted Testing have a time and place, and both test methods are needed.

We can use both of them without excluding one another. Using Scripted Testing doesn’t mean we can’t also use Exploratory Testing, and vice versa.

It’s just important to understand our goal to determine which method will meet our current needs.

Resources

A Practitioner’s Guide to Software Test Design by Lee Copeland