Attributing costs to test execution

How much does it cost to execute your tests?

The problem

On the face of it you may say, my test execution doesn’t cost anything. I run my tests on a build agent or Docker Container, and there are no costs incurred.

If you’re using Selenium Grid or other services, there may be some direct costs in terms of infrastructure (either self-hosted or paid cloud)

The costs which aren’t usually considered are:

• How long does it take for your tests to execute?
• How long are you waiting for the results?

And…

• If a test or multiple tests fail, what is your process?
• Do you re-run the build?
• Do you have a way to only re-run failed tests?
• Or do you manually execute the failed tests somewhere else?

All of this time adds up very quickly, a test suite that took 20 minutes to execute but had 5 flaky tests, has now added up to almost an hour of investigation and re-running of tests.

Read the full post here: https://garyparker.dev/blog/attributing-costs-to-test-execution