Software Testing Basics: Why Unit Tests Matter

Every time you change a program, you risk breaking something that worked yesterday. You can check by running the program and trying a few things by hand, but that gets slow and unreliable as the program grows. After the tenth change you stop checking everything, and the bug you missed ships. Automated tests fix this problem. They are small programs that check your program, and they can be run again in seconds.

This entry explains what a unit test is and why it is worth the effort. It then walks through writing and running real tests with Python's built-in unittest module, using a leap-year checker as the example. It assumes you can read a function and run a Python file.

What testing is for

Software testing means checking that a program does what it should. Tests are run at several levels, and the names help you talk about them:

This entry is about the first row. A unit test is a small automated check that runs one piece of code with known input and compares the result with the expected one. Unit tests are quick, they point at the exact function that failed, and they are the base that the other levels build on.

Tests give you three things:

• Early warning. A bug found by a test the minute you write it usually costs less than one a user finds later.
• Safety when changing code. If the tests still pass after you rewrite a function, you have good evidence that you did not break it. This protection against old bugs coming back is called regression testing.
• A description of intent. A test shows how a function is supposed to be used and what it should return. It is documentation that is much harder to leave out of date, because it fails when the behavior changes and the tests are run.

A function to test

Take a leap-year rule. A year is a leap year if it is divisible by 4, except that years divisible by 100 are not, unless they are also divisible by 400. Save this in a file named leap_year.py:

def is_leap_year(year):
    if year % 400 == 0:
        return True
    if year % 100 == 0:
        return False
    return year % 4 == 0

The % sign gives the remainder after division. 2024 % 4 is 0, so 2024 passes the last line and is a leap year. 1900 % 100 is 0 but 1900 % 400 is 300, so it stops at the second check and returns False.

Writing your first tests

Python ships with the unittest module. A test case is a class that inherits from unittest.TestCase, and each method whose name starts with test is one test. Save this as test_leap_year.py in the same folder:

import unittest
from leap_year import is_leap_year


class TestLeapYear(unittest.TestCase):
    def test_divisible_by_4_is_leap(self):
        self.assertTrue(is_leap_year(2024))

    def test_not_divisible_by_4_is_not_leap(self):
        self.assertFalse(is_leap_year(2023))

    def test_century_is_not_leap(self):
        self.assertFalse(is_leap_year(1900))

    def test_every_400_years_is_leap(self):
        self.assertTrue(is_leap_year(2000))


if __name__ == "__main__":
    unittest.main()

Each test has the same three-part shape, often called arrange, act, assert. Arrange the input, act by calling the function, and assert that the result is what you expect. The assert methods do the comparing. assertTrue and assertFalse check a boolean. assertEqual(a, b) checks that two values are equal and shows both values when they are not. assertRaises checks that the code raises a chosen kind of error, which is useful for testing how a function reacts to bad input.

To run the tests, open a terminal in that folder and type:

python -m unittest

This command finds the test files by name and runs them. With the correct function, the output looks like this (the timing will differ on your computer):

....
----------------------------------------------------------------------
Ran 4 tests in 0.001s

OK

Each dot is one passing test.

What a failing test looks like

A test earns its place when it fails for the right reason. Suppose a developer simplifies the function and forgets the century rule:

def is_leap_year(year):
    return year % 4 == 0

Running the tests again now reports a failure:

F...
======================================================================
FAIL: test_century_is_not_leap (test_leap_year.TestLeapYear.test_century_is_not_leap)
----------------------------------------------------------------------
...
AssertionError: True is not false

----------------------------------------------------------------------
Ran 4 tests in 0.005s

FAILED (failures=1)

The F marks the failed test, and its name says which rule broke: the century rule. The message says the function returned True where False was expected, because 1900 is divisible by 4. Three of the four tests still pass, which tells you exactly how far the damage goes. Finding the same problem by hand would take longer. Once you see the failure, the routine in how to debug code step by step applies. Restoring the missing line makes all four tests pass again.

Choosing good test cases

You cannot test every possible input, so choose cases that teach you something.

1. A normal case. An ordinary input with a clear answer, like 2024.
2. Edge cases. The boundaries where logic changes, such as 0, 1, the largest and smallest allowed values, an empty list or an empty string. The leap-year rules have three layers, so the four tests above cover one example from each layer.
3. Error cases. What should happen with bad input, such as text where a number belongs? Decide, then test it.
4. A bug you just fixed. Write a test that would have caught it. It keeps the bug from returning.

Keep each test small and give it a name that reads like a sentence, such as test_century_is_not_leap. When a test fails, the name should say what is wrong before you open the file.

Habits that make tests useful

• Test behavior, not internal steps. A test should say what the function returns, not how it reaches the answer. Then you can rewrite the inside freely.
• Keep tests independent. One test should never depend on another having run first.
• Run them often. Run the tests after every small change. The sooner a failure appears, the fewer lines you have to search.
• Make them fast. Slow tests get skipped. Unit tests should finish in moments.
• Write the test first when you can. Some developers write a failing test, then the code that makes it pass, then tidy up. This approach is called test-driven development. It is optional, but trying it once shows how tests shape design.

Tests are also easier to write when functions are small and have clear inputs and outputs, which is one more reason to follow the advice in the entry on what an algorithm is and keep steps well defined. More topics are collected in the Logic and Programming area.

Common mistakes

• Testing only the happy path. If every test uses a normal input, bugs at the edges survive. Add boundary and error cases.
• Tests that always pass. A test with no assertion, or one that checks the wrong thing, gives false confidence. Break the code on purpose once to see the test fail.
• Copying the code's logic into the test. If the test uses the same formula as the function, a wrong formula passes both. Use known answers from outside, such as dates you can check on a calendar.
• Depending on hidden state. Tests that read the current date, random numbers or a shared file can pass today and fail tomorrow. Give the function fixed input.
• Ignoring a failing test. Commenting out a red test hides the problem. Fix the code or fix the test, and know which one was wrong.
• Chasing perfect coverage. The goal is confidence in the parts that matter, not a test for every line.

FAQ

What is the difference between a unit test and an integration test?

A unit test checks one small piece alone. An integration test checks that several pieces work together, such as code and a database. Both are useful, and unit tests are usually the more numerous because they are quick.

Do I need a testing framework?

You can write a plain check yourself, but frameworks such as unittest give you assertions, readable failure messages and a way to run everything at once. Python also has other popular choices, and the ideas carry over between them.

Does a passing test prove the code is correct?

No. It proves the code works for the inputs that you tested. That is why the choice of cases matters.

How many tests should a function have?

Enough to cover each rule and each edge in it. A function with three rules, like the leap-year check, needs at least one test per rule.

Is testing the same as debugging?

They are related but different. Testing finds that something is wrong. Debugging finds why and fixes it. A good test makes the debugging step much shorter.