The wisdom of the crowd has created an awesome resource for context-driven testers

Published on August 17, 2024

Background

I’ve enjoyed a long and fruitful relationship with the Association for Software Testing (AST), both as a member and through the CAST conferences (where I’ve been an organizer for Australian conferences and a speaker & delegate at US conferences).

When I was approached about the idea of creating an e-book to provide responses to some common questions and statements about testing from a context-driven perspective, I was keen to be involved. The idea was to crowdsource the content and then for me to collate responses for the e-book from this content. The e-book was designed to act as an FAQ for the day-to-day situations a tester may find themselves in and how to approach them from a context-driven perspective.

The Navigating the World as a Context-Driven Tester e-book project kicked off early in 2021 and the final edit has just been made after some 28 requests for contributions. These requests were made across multiple platforms, including Twitter (X), LinkedIn, Slack (viz. in the AST and Rapid Software Testing channels), Mastodon and the AST mailing list.

My experience of putting the book together

I loved the concept for this e-book and was excited to make the first few requests for contributions to see if there was similar interest and excitement in the project from the broader testing community. I was pleased to see lots of early engagement and my job in collating a response for the book was often a difficult one thanks to the sheer number and diversity of responses received.

It was interesting to see which requests got the most interest and I was often surprised by which ones received many responses. It was great to see that responses continued to come in even as the project entered its third year.

The best channels to elicit responses from requests for contributions to the e-book changed over time.

The AST and RST Slack channels provided by far the largest numbers of responses across the project, perhaps reflecting the community of more seasoned practitioners active on these platforms. Twitter was a good source at the start of the project, but faded quickly as many testers moved off the X platform. LinkedIn was fairly consistent throughout, but never a huge source of responses. The inclusion of Mastodon for the last year or so of the project resulted in only a very small number of responses and the AST newsletter was similarly ineffective in generating responses.

This e-book has been compiled from the collective wisdom of many excellent testing practitioners and I feel that it book provides a lot of value, especially to less experienced testers. It is my hope that the e-book will be a handy reference for testers and I look forward to hearing stories of how it’s proved to be useful.

As a reminder, the e-book is freely available on the AST’s GitHub, Navigating the World as a Context-Driven Tester.

Vital statistics

A few stats about the book and the process of creating it:

  • 72 contributors helped to shape the content of the e-book (all of whom are attributed).
  • 292 responses were received from the 28 requests for contributions.
  • The request that drew the most interest was “Testing is a bottleneck”, with 28 responses.
  • James Thomas provided responses to all 28 requests via a separate blog post for each request. Amit Wertheimer and Frances Turnbull chipped in more than 20 responses each.
  • 7 requests for contributions were made in 2021, 9 in 2022, 7 in 2023 and 5 in 2024.
  • The 28 responses consist of 7 in the “Testing” category, 6 in “Testers”, 4 in “Automation”, 3 in “Context-driven testing”, 3 in “Project Scheduling”, 2 in “Testing Status”, 2 in “Career” and 1 in “Scripts/test cases”.
  • The requests that drew the least interest were “What’s the best format for a test plan?” and “Pair and ensemble testing look like a waste of time and resources to me. What do you think?”, with just 4 responses each.

With thanks

Thanks to the AST for trusting me with the curation of this project and also to the various AST board members who reviewed my collated responses before publication and supported the project in numerous ways.

I’m so grateful to the 72 folks who made the effort to contribute responses – this book wouldn’t exist without you!

The questions/statements

The questions/statements that formed the 28 requests for contributions are listed below:

Question/statement Request made #responses response published
We test to make sure it works 07/04/21 21 21/06/21
Let’s just automate the testing 03/05/21 9 21/06/21
Isn’t all testing context-driven? 14/06/21 15 19/07/21
Do more test cases mean better test coverage? 27/07/21 13 29/08/21
What percentage of our test cases are automated? 08/09/21 9 03/10/21
Stop saying “it depends” when I ask you a question 11/10/21 13 01/11/21
Testing is a bottleneck 06/12/21 28 10/01/22
What’s the difference between context-driven testing and exploratory testing? 16/01/22 6 12/02/22
Will the testing be done by Friday? 11/02/22 8 07/03/22
We need some productivity metrics from testers 12/03/22 9 16/04/22
There are no best practices, really? 01/05/22 5 26/05/22
Why didn’t you find those issues before we shipped? 05/06/22 5 02/07/22
For your annual review, I’ll need to see evidence of what you produced this year 01/08/22 12 23/08/22
What’s the right ratio of developers to testers? 27/08/22 10 27/09/22
What’s the best testing metric? 03/10/22 8 07/10/22
Testing is just to make sure the requirements are met 03/11/22 15 04/12/22
Whenever possible, you should hire testers with testing certifications 11/01/23 16 06/02/23
Developers can’t find bugs in their own code 12/02/23 10 13/03/23
Stop answering my questions with questions 18/03/23 9 16/04/23
When is the best time to test? 29/04/23 10 22/05/23
Testers are the gatekeepers of quality 29/05/23 15 12/07/23
If testers can’t code, they’re of no use to us 18/07/23 9 15/08/23
Is observability and monitoring part of testing? 11/11/23 9 06/12/23
When the build is green, the product is of sufficient quality to release 08/01/24 7 07/02/24
What’s the best format for a test plan? 22/02/24 4 21/03/24
Why don’t we replace the testers with AI? 07/04/24 7 29/04/24
How can I possibly test “all the stuff” every iteration? 01/06/24 6 08/07/24
Pair and ensemble testing look like a waste of time and resources to me. What do you think? 23/07/24 4 16/08/24

(The featured image for this post was inspired by my recent travels to Lisbon, Portugal, and its famous number 28 tram – thanks to Victoria Emerson on Pexels.com)

number-28