Testers, Code and Automation, Part 2

This post will continue from the first post. Here I’ll continue the exploration of testers interacting and intersecting with code, while making sure to consider good test and code practices.

As mentioned in the first post, there is a GitHub repo that I’m providing in case someone wants to use that. It’s tagged at various points and I’ll indicate what tags you can checkout if you want to join in at a certain place. Case in point, if you want to start off where we ended:

git checkout tags/firstpost

To set the stage for the goal of this post, I’m going to refine the code from the first post. Keep in mind that in the initial implementation, while we had a prime number calculator, there was no actual user interface. Here, I’m going to use a bit of a contrived example to allow the user to enter input into the prime calculator. Once the user does so, the logic will fall through to what we looked at in the last post.

In the previous post our main function looked like this:

func main() {
	data := 2

  _, outcome := checkPrime(data)

	fmt.Println(outcome)
}

Let’s say our developers are going change that in order to add the ability for a user to input a number that then gets passed to our checkPrime function. Regarding that function, let’s revisit the feature code we ended up with:

func checkPrime(data int) (bool, string) {
	if data <= 1 {
		return false, fmt.Sprintf("%d is not prime", data)
	}

	for i := 2; i*i <= data; i++ {
		if data%i == 0 {
			return false, fmt.Sprintf("%d is not prime; divisible by %d", data, i)
		}
	}

	return true, fmt.Sprintf("%d is prime", data)
}

As a reminder, that can be run with:

go run .

The tests we wrote so far can be executed as such:

go test .

And if you want to get coverage measures, you can do this:

go test -cover .

Updating the Feature

Our developers go to work and start refactoring main and creating some new logic:

func main() {
	startup()
}

func startup() {
	fmt.Println("Is number prime?")
	fmt.Println("Enter a whole number; q to quit.")
	prompt()
}

func prompt() {
	fmt.Print("> ")
}

This is probably pretty self-explanatory but I’m starting slow so readers can follow the evolution of the code. The developers continue to build out the main function:

func main() {
	startup()

	exitChannel := make(chan bool)

	go getInput(exitChannel)

	<-exitChannel

  close(exitChannel)

  fmt.Println("Exiting")
}

func getInput(exitChannel chan bool) {}

This creates what’s called a channel of boolean type. This channel will be used to signal the end of the program from a goroutine that I’m calling getInput.

Think of goroutines as lightweight tasks that can be easily created and managed. The Go runtime acts as a smart traffic controller, efficiently scheduling these tasks onto a smaller number of actual “roads” (operating system threads). The Go runtime executes goroutine functions concurrently with other parts of your program.

What this logic does is start a new goroutine by calling the getInput function and passing the exitChannel to it. The seemingly odd line <-exitChannel blocks the main function until it receives a value from the exitChannel. This ensures that the main function waits for getInput to signal its completion. The call to close does what it sounds like: it closes the exitChannel to prevent further sends on it.

A channel is a communication conduit for goroutines. Think of it like a pipe. Goroutines can send and receive data through this pipe. Channels provide a way for goroutines to synchronize their execution, ensuring that data is passed correctly between them.

Now, as a tester, you might look at this developer code and wonder if we aren’t complicating things a wee bit. After all, do we really need to use channel pipes and concurrent goroutines for something like this? No, we don’t. There will be a point to this as we move on.

I’m indebted to Trevor Sawler and his Udemy course “Introduction to Testing in Go” for the idea of creating a perhaps overly contrived example of user input.

The developers now fill out that getInput function.

func getInput(exitChannel chan bool) {
	scanner := bufio.NewScanner(os.Stdin)

	for {
    result, done := getNumber(scanner)

		if done {
			exitChannel <- true
			return
		}

		fmt.Println(result)
		prompt()
	}
}

func getNumber(scanner *bufio.Scanner) (string, bool) {
	return "", true
}

Here, the developers have created a scanner to read input from the standard input (keyboard). This logic enters an infinite loop to continuously read and process input. A call to the getNumber function — currently just a stub — will process the input and get a result string and a boolean flag indicating whether to exit.

If done is true — which would mean that the user entered “q” to quit — the logic sends a true value on the exitChannel to signal the main function and then returns from the getInput routine.

Finally, the developers fill out the getNumber function.

func getNumber(scanner *bufio.Scanner) (string, bool) {
	scanner.Scan()

	if strings.EqualFold(scanner.Text(), "q") {
		return "", true
	}

  number, err := strconv.Atoi(scanner.Text())

	if err != nil {
		return "Enter a whole number", false
	}

  _, outcome := checkPrime(number)

  return outcome, false
}

This logic reads a line of input from the scanner. Assuming a “q” was not entered, the logic attempts to convert the input string to an integer. If the conversion fails — meaning the input is not a number — it returns an error. Note that here, finally, is where we call our previous checkPrime function.

The developers claim they are done for now so let’s take a look at our full source code and thus the updated feature.

package main

import (
	"bufio"
	"fmt"
	"os"
	"strconv"
	"strings"
)

func main() {
	startup()

	exitChannel := make(chan bool)

	go getInput(exitChannel)

	<-exitChannel

	close(exitChannel)

	fmt.Println("Exiting")
}

func getInput(exitChannel chan bool) {
	scanner := bufio.NewScanner(os.Stdin)

	for {
		result, done := getNumber(scanner)

		if done {
			exitChannel <- true
			return
		}

		fmt.Println(result)
		prompt()
	}
}

func getNumber(scanner *bufio.Scanner) (string, bool) {
	scanner.Scan()

	if strings.EqualFold(scanner.Text(), "q") {
		return "", true
	}

	number, err := strconv.Atoi(scanner.Text())

	if err != nil {
		return "Enter a whole number", false
	}

	_, outcome := checkPrime(number)

	return outcome, false
}

func startup() {
	fmt.Println("Is number prime?")
	fmt.Println("Enter a whole number; q to quit.")
	prompt()
}

func prompt() {
	fmt.Print("> ")
}

func checkPrime(data int) (bool, string) {
	if data <= 1 {
		return false, fmt.Sprintf("%d is not prime", data)
	}

	for i := 2; i*i <= data; i++ {
		if data%i == 0 {
			return false, fmt.Sprintf("%d is not prime; divisible by %d", data, i)
		}
	}

	return true, fmt.Sprintf("%d is prime", data)
}

git checkout tags/userinput

Obviously you can run that logic and, to test it, enter in a series of data conditions.

go run .

Here is a usage scenario:

  Is number prime?
  Enter a whole number; q to quit.
  > 0
  0 is not prime
  > 1
  1 is not prime
  > -1
  -1 is not prime
  > 4
  4 is not prime; divisible by 2
  > 7
  7 is prime
  > q
  Exiting

And equally obviously you could rerun that set of tests by hand every time a change was made to the code.

Remember, however, that we have the tests wrote in the previous post. We ended that post with just over 60% coverage, which was covering all the logic of the checkPrime function. But we’ve added a lot of new code here. So let’s see what the situation is now.

go test -cover .

You’ll likely get a coverage of only 18.8%. As before, we can check out what’s missing in the browser view.

go test -coverprofile="coverage.out" .
go tool cover -html="coverage.out"

You will see that the checkPrime function is still tested at 100% due to the tests that we wrote in the previous post. So, the only thing not being tested is all the new code. Essentially, we’re missing the entirety of the new feature that allows a user to specify the number to check!

What To Test First?

Let’s first add a test for the prompt function. But isn’t this a bit of a silly thing to write a test for? After all, the prompt function is literally one line of code. While that’s true, this test will actually help us understand how to write a test to look at the output delivered to the terminal.

So this is a test that, while perhaps only having debatable overall value, is helping us better understand how to construct a code-based test.

We have our main_test.go file from the previous post, so we add a test for the prompt to that:

func Test_prompt(t *testing.T) {
	currentOut := os.Stdout

	readPipe, writePipe, _ := os.Pipe()

	os.Stdout = writePipe

	prompt()

	_ = writePipe.Close()

	os.Stdout = currentOut

  terminalOutput, _ := io.ReadAll(readPipe)

	if string(terminalOutput) != "> " {
    t.Errorf("incorrect prompt; expected > | got %s", string(terminalOutput))
	}
}

What this does is redirect the standard output stream to the writePipe. This means that any output that would normally be printed to the console will now be written to the pipe instead. It then reads all the data written to the pipe.

Let’s run the tests.

go test .

This will pass, perhaps not surprisingly. Now here’s an example of where as a test writer, we might want to consider dependency injection. This is going to put pressure on design. If the prompt function were refactored to accept an io.Writer as a parameter, testing would be simpler and more reliable. For example, we could encourage the developers to change the prompt function like this:

func prompt(out io.Writer) {
    fmt.Fprint(out, "> ")
}

Note all the changes there! For such a simple function, that’s actually a relatively significant set of changes. In fact, every part of the function is changed! This change now requires that the developers change the calls to prompt. In getInput and in startup, the call becomes:

prompt(os.Stdout)

With this in place, the test we just wrote becomes much simpler:

func Test_prompt(t *testing.T) {
    var buf bytes.Buffer

    prompt(&buf)

    if buf.String() != "> " {
        t.Errorf("incorrect prompt; expected '>' | got '%s'", buf.String())
    }
}

This approach enhances robustness, readability, and flexibility. This required some refactorings in our code logic, however. That should raise your coverage a bit, to about 21.9%

Was this extra bit of coverage worth it? Well, for those who are in service of one hundred percent code coverage, perhaps. Even without that, however, as a tester something like a prompt can also become “background noise.” It would be easy for the prompt to change and you might not notice. The code-based test won’t fall prey to that.

As with most intersections of human testing and code-base tests, it’s not a case of “either-or”; it’s a case of “both-and.” Human and automation working together. Also notice what attempting to test the prompt did? It put pressure on our design.

Next Test

Now let’s test the startup function. This is going to be very similar to what we did above. We can add this test:

func Test_startup(t *testing.T) {
 	var buf bytes.Buffer

 	startup()

 	if !strings.Contains(buf.String(), "Enter a whole number") {
 		t.Errorf("incorrect startup; got '%s'", buf.String())
 	}
}

However, if you run this, the test will fail:

> --- FAIL: Test_startup (0.00s)
    main_test.go:64: incorrect startup; got ''

The problem here is that I’m setting up the buf variable but I’m not actually passing that to the startup. Yet, I can’t do that because our code isn’t set up to do that! To run this test, we have to work with the developers to change the startup signature:

func startup(out io.Writer) {

Since main calls startup, that has to be updated as well:

func main() {
  startup(os.Stdout)
  ...

Now let’s update the test:

func Test_startup(t *testing.T) {
	var buf bytes.Buffer

	startup(&buf)

	if !strings.Contains(buf.String(), "Enter a whole number") {
		t.Errorf("incorrect startup; got '%s'", buf.String())
	}
}

The developers would actually find they have a problem here with startup.

This means they have to refactor that logic to this:

func startup(out io.Writer) {
	fmt.Fprintln(out, "Is number prime?")
	fmt.Fprintln(out, "Enter a whole number; q to quit.")
	prompt(out)
}

Okay, and that finally got it! You should be able to run the tests and they will pass. Your coverage should now be about 31.2%.

Notice how much refactoring we ended up doing as a result of tests. Yet, also notice how easy it was since we worked on keeping the code we were testing light in terms of implementation. Now a good question for someone to ask is: while this put pressure on our design, did it actually improve our design?

Speaking not to just to the design of our code, let’s also consider the design of our last test. It only tests a portion of what the startup function displays. We might want it to be a little more comprehensive and test all of the startup text:

func Test_startup(t *testing.T) {
	var buf bytes.Buffer

	startup(&buf)

	output := buf.String()

	expectedLines := []string{
		"Is number prime?",
		"Enter a whole number; q to quit.",
	}

	for _, line := range expectedLines {
		if !strings.Contains(output, line) {
			t.Errorf("missing expected output: %q; got '%s'", line, output)
		}
	}
}

Notice how even though we are now testing more of the actual output, our coverage remains the same: 31.2%. This is an important distinction between code coverage and test coverage.

It’s worth noting that while we now fully test prompt and startup, we don’t cover getInput or getNumber yet at all. That’s something we’ll do in the third, and final, post to this series.