Blocking Bugs and Building Quality Software with the Test Pyramid

In my first article, “ test case design mindset at a glance”, I discussed how to approach test case creation to ensure quality, using the Place Order API and a volleyball analogy. While that article focused on finding bugs, in today’s fast-paced software development environment, preventing bugs has become even more crucial. Choosing the right testing strategy is key to ensuring product quality, speeding up releases, and avoiding costly bugs in production.

One well-known approach to balanced testing is the Test Pyramid, popularised by Mike Cohn in his 2009 book Succeeding with Agile: Software Development Using Scrum. This concept emerged from discussions in the early 2000s, with contributions from figures like Martin Fowler and Jason Huggins, who explored similar ideas independently.

In this article, I will build on my previous discussion, again using the Place Order API and volleyball analogy to explain key concepts of the Test Pyramid in software development, with practical examples to illustrate each layer.

What is the test pyramid ?

The Test Pyramid is a testing strategy that consists of different layers of tests, arranged like a pyramid, and integrated into the software deployment pipeline. As we move up each layer of the pyramid, the tests differ in scope, speed and cost:

1. Scope: The breadth of functionality the tests cover.
2. Speed: How quickly the tests execute and how fast bugs can be identified and fixed.
3. Cost: The resources required to maintain the tests and resolve any bugs.

As we move up the pyramid, the number of tests decreases because higher-level tests take longer to execute and cover a broader scope, which increases complexity and cost. This is why the base of the pyramid (unit tests) contains the majority, being fast and inexpensive, while the top (end-to-end tests) has fewer, more complex tests.

Unit tests

Unit tests are the foundation of the Test pyramid, designed to test individual pieces of code in isolation. They focus on ensuring that small, specific functions or methods behave as expected.

Below is an example of unit tests for calculate_order_total function in a place order service:

# place_order_api.py (API endpoint for placing an order)

def calculate_order_total(cart_items):
    """Function to calculate the total cost of items in the cart"""
    total = 0
    for item in cart_items:
        total += item['price'] * item['quantity']
    return total

# test_order.py (Unit Test)
import unittest
from order import calculate_order_total

class TestPlaceOrder(unittest.TestCase):
    def test_calculate_order_total(self):
        cart_items = [
            {'name': 'Volleyball', 'price': 1000, 'quantity': 10},
            {'name': 'Ball Cart', 'price': 50, 'quantity': 1}
        ]
        total = calculate_order_total(cart_items)
        self.assertEqual(total, 10050)

    def test_negative_price(self):
        cart_items = [
            {'name': 'Volleyball', 'price': -1000, 'quantity': 10}
        ]
        total = calculate_order_total(cart_items)
        self.assertEqual(total, -10000)

if __name__ == '__main__':
    unittest.main()

Now, let’s think of it in volleyball terms. A unit test is like a player practicing individual skill, such as serving, passing, setting or spiking. These are the basic building blocks of the game. Each player focuses on mastering their own technique before coming together as a team.

Similarly, a unit test focuses on a small, isolated part of the code to ensure it performs correctly, just as practicing serves ensures the player is consistently accurate.

Integration tests

Sitting above unit tests, integration tests verify how different components or services interact with each other. They are crucial for identifying issues that may not arise within isolated units but the agreed format of communication between two services. (provider and consumer)

For instance, integration tests for the /place_order API ensure that the order processing logic interacts correctly with the external payment service, preventing errors that could arise from mismatched expectations:

# place_order_api.py (API endpoint for placing an order)

def process_payment(payment_details, amount):
    """Function to call an external payment service"""
    payment_service_url = "http://external-payment-service.com/api/process-payment"
    
    payment_payload = {
        "card_number": payment_details['card_number'],
        "amount": amount
    }
    
    response = requests.post(payment_service_url, json=payment_payload)

    if response.status_code == 200:
        return True
    return False

@app.route('/place-order', methods=['POST'])
def place_order():
    data = request.get_json()
    cart_items = data.get('cart_items')
    payment_details = data.get('payment_details')

    # Calculate the total order amount
    total_amount = calculate_order_total(cart_items)

    # Check if the payment can be processed
    if not process_payment(payment_details, total_amount):
        return jsonify({"message": "Payment failed"}), 400
    
    # Order successfully placed
    return jsonify({"message": "Order placed successfully", "order_id": 123, "total_amount": total_amount}), 200

if __name__ == '__main__':
    app.run()

# test_order_api.py (Integration Test for the Place Order API)
import requests
import unittest

class TestPlaceOrderAPI(unittest.TestCase):
    def test_place_order_success(self):
        response = requests.post('http://localhost:5000/place-order', json={
            'cart_items': [{'name': 'Volleyball', 'price': 1000, 'quantity': 10}],
            'payment_details': {'card_number': 'valid'}
        })
        self.assertEqual(response.status_code, 200)
        self.assertEqual(response.json().get('message'), 'Order placed successfully')

    def test_place_order_payment_failed(self):
        response = requests.post('http://localhost:5000/place-order', json={
            'cart_items': [{'name': 'Ball Cart', 'price': 50, 'quantity': 1}],
            'payment_details': {'card_number': 'invalid'}
        })
        self.assertEqual(response.status_code, 400)
        self.assertEqual(response.json().get('message'), 'Payment failed')

if __name__ == '__main__':
    unittest.main()

Imagine a volleyball team where each player has a specific role — setters deliver the ball to hitters, and liberos play defensively. The success of a play depends on how well these roles integrate; if the setter’s delivery is off, the hitter can’t execute a successful attack. This is about testing teamwork and coordination to ensure the team can work together effectively. Much like how a software application relies on different components working together seamlessly.

End-to-end tests

At the top of the testing pyramid are end-to-end tests. which provide the team with the greatest confidence by stimulating actual user journeys from start to finish.

For instance, in the place order flow, an end-to-end test would simulate a user journey starting from adding items to a cart, proceeding to checkout, entering payment details, and confirming the order.

// place_order.spec.js (E2E Test using Playwright)
const { test, expect } = require('@playwright/test');

test.describe('Place Order Flow', () => {
  
  test('should successfully place an order', async ({ page }) => {
    // Navigate to the e-commerce site
    await page.goto('http://localhost:3000');

    // Add item to the cart
    await page.click('text=Volleyball');
    await page.click('text=Add to Cart');

    // Navigate to the cart page
    await page.click('text=Cart');

    // Proceed to checkout
    await page.click('text=Checkout');

    // Fill in shipping and payment details
    await page.fill('#shipping-address', '123 Main St, City, Country');
    await page.fill('#card-number', 'valid');

    // Place the order
    await page.click('text=Place Order');

    // Assert the success message
    await expect(page.locator('text=Order placed successfully')).toBeVisible();
  });

Finally, in volleyball, the ultimate test of skill is playing a full match. Here, everything comes together — serving, passing, setting, spiking, blocking — under real match conditions. This scenario reveals how well players perform under pressure, showcasing the effectiveness of their strategies and teamwork in a real game.

Now that we understand the Test Pyramid, let’s compare it side by side with the Ice-Cream Cone anti-pattern:

What is the ice cream cone?

A common anti-pattern to avoid is the ice-cream cone, the opposite of the pyramid. Its narrow base makes it unstable, resulting in slow test execution and high maintenance. With most tests concentrated at the top, managing them becomes difficult — much like trying to hold a Turkish ice cream cone from the vendor.

An example of this anti-pattern is an over-reliance on full end-to-end order flows that bundle payment, shipping, and notifications into a single, slow-running test. This approach results in limited tests that verify crucial interactions between the order and payment services, creating potential gaps in system communication. Furthermore, there are minimal unit tests for basic functions, such as calculating order totals, leaving essential functionality untested and increasing the risk of undetected bugs.

Imagine a volleyball team that only plays full matches, focusing solely on the entire game with little attention to individual skills. With minimal focus on passing drills or setting techniques, critical areas of the game are neglected. As a result, the team may struggle with basic skills, leading to underperformance during matches.

Similarly, in software development, relying only on end-to-end tests without enough unit tests leaves core functionality untested, potentially causing issues that could have been easily caught early.

Summary:

Adopting a structured testing strategy, such as the test pyramid, is essential for ensuring software quality and maintaining efficient workflows. While both unit tests and end-to-end tests provide valuable coverage and enhance user experiences, it’s crucial to carefully structure our testing approach to achieve balance and efficiency.

To further illustrate these concepts, we can draw parallels with volleyball.

Unit Tests = Practicing Individual Skills: Quick, easy, focused on small tasks like serving or passing. In testing, it’s checking small parts of code.

Integration Tests = Team Drills: Checking how players work together. In testing, it’s ensuring different parts of the system communicate well (like passing between payment and order).

E2E Tests = Full Game: Competing in an entire match. In testing, it’s running the software as the user would, from start to finish, to ensure everything works smoothly.

Different projects, technologies, and team structures call for different approaches. There isn’t a one-size-fits-all approach. Engineers should work together to identify the most effective testing strategy that distributes tests across different layers, allowing issues to be caught earlier in the development cycle. By understanding these concepts, we can better navigate our testing strategies and improve our overall software quality. (And volleyball game)

Thanks for reading and lets continue to share and make your job as fun as your game.

Blocking Bugs and Building Quality Software with the Test Pyramid was originally published in Government Digital Products, Singapore on Medium, where people are continuing the conversation by highlighting and responding to this story.