Golang: How to Test Code That Exits or Crashes?

Today I'll go over how to write "exitable/crasheable" tests in the Go language. This question emerged when I attempted to write my tests in the same way that I did in other languages. Before I show you how I develop some tests, I'd want to share some useful code design tips.

Go error handing ❌

My first language experience was procedural, and then I moved on to object-oriented programming, so on the Go side, we have rules to handle errors in order to avoid problems during pipeline execution and testing; thus, with my limited Go skills, I would argue that you should not use panic functions at all or log.Fatal, which results in an os.exit since you are interrupting the execution of your program, and you may not want this behavior to be shared across all of your packages.

Assume you're consuming a package that could crash, and your system isn't expecting it, so every crash means a worker, server, or pod is offline. If we are working with a http server, it will most likely send an internal server error for this piece of code is isolated. Of course, you can handle issues at the error entry point, but I believe this is not the best solution because it adds 🦨 smell to your code. This is equivalent to failing to separate your ♻️ recyclable garbage.

I understand that return error for all functions can be difficult to handle code; at Elixir, we have pattern matching to deal with error verbosity, but again, return error is much better than panic or interrupting; explicit is better than implicit; for a long time, I used to prefer less magic than the past; I used to love the way Rails, Django, ASP.NET, and Spring does things, but after lost nights 💤 and much coffee ☕, I choose simplicity at some places of code design.

Writing a calculator

Let's develop a calculator 🧮 using the panic and exit functions to show how program interruption can be troubling during testing.

• main.go

package main

import (
	"log"
	"os"
	"strconv"
)

// This function is a minimal implementation of the calculator
func calculate(args []string) float64 {
	if len(args) < 3 {
		panic("invalid arguments")
	}

	x, err := strconv.Atoi(args[0])

	if err != nil {
		panic(err)
	}

	y, err := strconv.Atoi(args[2])

	if err != nil {
		panic(err)
	}

	var r float64

	switch args[1] {
	case "+":
		r = float64(x + y)
	case "-":
		r = float64(x - y)
	case "x":
		r = float64(x * y)
	case "/":
		r = float64(x / y)
	default:
		log.Fatal("invalid operation")
	}

	return r
}

func main() {
	args := os.Args[1:]

	r := calculate(args)

	log.Printf("🟰  %.2f\n", r)
}

• Panic functions are used when numbers cannot be parsed or the required arguments are less than three in length.
• When an operation does not exist, the program sends a log.Fatal that causes os.exit.

Then let's run the application.

# installing deps
go mod download

go run main.go 2 - 9
# 🟰 -7

go run main.go 5 + 2
# 🟰 7

go run main.go 7 x 7
# 🟰 49

go run main.go 49 / 7
# 🟰  7.00

Before writing tests, we need some functions to support them.

• test.go

package main

import (
	"bytes"
	"fmt"
	"os"
	"os/exec"
	"testing"
)

// Run a fork test that may crash using os.exit.
func RunForkTest(t *testing.T, testName string) (string, string, error) {
	cmd := exec.Command(os.Args[0], fmt.Sprintf("-test.run=%v", testName))
	cmd.Env = append(os.Environ(), "FORK=1")

	var stdoutB, stderrB bytes.Buffer
	cmd.Stdout = &stdoutB
	cmd.Stderr = &stderrB

	err := cmd.Run()

	return stdoutB.String(), stderrB.String(), err
}

• The RunForkTests function runs a specified test in a fork process and allows you to assert stdout and stderr.

So right now is the time for coding tests

💡Okay, we've reached the main goal: create crashable tests. The most common return execution status is frequently 0 for success and 1 for error. This status indicates whether the test was successful or failed.

To avoid test crashes caused by panic or os.exit, tests will run in a fork process. When a crash happens, the fork process terminates and the main progress matches the progress status and the related stdout and stderr files.

• main_test.go

package main

import (
	"bytes"
	"log"
	"os"
	"testing"

	"github.com/stretchr/testify/assert"
)

func TestCalculateSum(t *testing.T) {
	r := calculate([]string{"5", "+", "5"})

	assert.Equal(t, float64(10), r)
}

func TestCalculateSub(t *testing.T) {
	r := calculate([]string{"5", "-", "15"})

	assert.Equal(t, float64(-10), r)
}

func TestCalculateMult(t *testing.T) {
	r := calculate([]string{"10", "x", "10"})

	assert.Equal(t, float64(100), r)
}

func TestCalculateDiv(t *testing.T) {
	r := calculate([]string{"100", "/", "10"})

	assert.Equal(t, float64(10), r)
}

func TestCalculateWithPanic(t *testing.T) {
	defer func() {
		err := recover().(error)
		if err != nil {
			assert.Contains(t, err.Error(), "parsing \"B\": invalid syntax")
		}
	}()

	calculate([]string{"10", "/", "B"})

	t.Errorf("😳 The panic function is not called.")
}

func TestMainWithPanicWithFork(t *testing.T) {
	if os.Getenv("FORK") == "1" {
		calculate([]string{"A", "/", "10"})
	}

	stdout, stderr, err := RunForkTest(t, "TestMainWithPanicWithFork")

	assert.Equal(t, err.Error(), "exit status 2")
	assert.Contains(t, stderr, "parsing \"A\": invalid syntax")
	assert.Contains(t, stdout, "FAIL")
}

func TestMainWithExit(t *testing.T) {
	oldStdout := os.Stdout
	oldArgs := os.Args

	var buf bytes.Buffer
	log.SetOutput(&buf)

	defer func() {
		os.Args = oldArgs
		os.Stdout = oldStdout
	}()

	os.Args = []string{"", "10", "x", "10"}
	main()

	log.SetOutput(os.Stderr)

	assert.Contains(t, buf.String(), "🟰  100.00")
}

This is the test execution 🔥.

• I employ the package gotestfmt to produce test experiences similar to those we have in other languages. Remember, we develop code for humans, therefore colors and emoji are extremely important 🙄.

Code

💡 Feel free to clone this repository, which contains related files:

go-recipes/crashable-tests at main · williampsena/go-recipes

Contribute to williampsena/go-recipes development by creating an account on GitHub.

GitHubwilliampsena

That's all folks

In this article, I describe how to handle tests that use panic or os.exit; however, I recommend you avoid using this behavior in all places of code; instead, prefer return error:

package main

type func calculate(args []string) (float64, error)

Let the main responsible to panic or exit the application and writing these types of tests can be difficult.

Please provide feedback so that we can keep our 🧠 kernel up to date.

I hope I can assist you with writing tests or resolving your worries about structuring your code for error handling, as well as remind you to stay focused on your goal:

🕊️ "Many are the plans in the mind of a man, but it is the purpose of the Lord that will stand" . Proverbs 19:21.