Build with Python

Find the Bug

11 of 24LabDictionaries and SetsBuild a Game
On this page

Find the Bug

Every program you write will be wrong at first. That is not a failure of talent; it is how programming goes, for everyone. What separates people who get programs working from people who get stuck is method: a way of finding out where a program stops matching what you meant, and of proving afterwards that it is fixed. This module teaches that method. There is no new Python in it except one small statement, assert.

Before this, you can: trace a program with functions, loops and lists; write a function from its contract; choose a list, dictionary or set.

By the end you will be able to read an error report from the bottom, tell the common kinds of error apart, find the first value that is not what you expected, state what must be true with assert, choose the test cases that matter, and write the test that fails before you fix the bug.

Before you read on, decide what this prints:

def describe(n):
    if n > 100:
        return "big: " + str(number)
    return "small"

print(describe(5))
print(describe(50))

small
small

Nothing is wrong, apparently. But line 3 uses a name, number, that does not exist; the parameter is called n. Python has not noticed, because line 3 has not run: no call has passed a number above 100. A language that checks names and types before a program runs, as Java does, would have rejected this one. Python raises the error only when that line is reached, so a program can work for ten inputs and stop on the eleventh. The cases you choose to test decide what you find.

Crashes and wrong answers

Bugs come in two kinds, and they call for different first moves.

A crash stops the program with a message. It is the easier kind: Python tells you which line it was running and what went wrong there.

A wrong answer is a program that runs to the end and prints the wrong thing. Nothing tells you where. You have to find the place yourself, and the rest of this lesson is mostly about how.

A third thing is worth separating from both: a program that is wrong and has not shown it yet, like the one above. You find those only by running the cases that reach them.

Reading a traceback

When a program stops inside a function that was called by another function, Python reports the whole chain of calls. This report is a traceback. Here is a program that stops. Under it is the short form of the report, and then the full report that Python gives when you run the program outside the lab:

def share(total, people):
    return total / people

def report(total, groups):
    for people in groups:
        print(share(total, people))

report(12, [3, 0])

  File "your_program.py", line 2
    return total / people
ZeroDivisionError: division by zero
Traceback (most recent call last):
  File "your_program.py", line 8, in <module>
    report(12, [3, 0])
  File "your_program.py", line 6, in report
    print(share(total, people))
  File "your_program.py", line 2, in share
    return total / people
ZeroDivisionError: division by zero

Read it from the bottom. The last line says what went wrong: a division by zero. The lines just above it say where: line 2, inside share. Above that is the line that called share, and above that the line that called report. The top of the report is where the program started; the bottom is where it died.

The lab shows you the bottom of this: the kind of error, the message and the line. When you step through the program instead, the boxes of calls in progress at the moment it stops are the same chain, drawn.

The line where a program stops is where the problem was noticed, which is not always where the mistake is. Line 2 here is a perfectly good line. The mistake is the 0 in the list on line 8, or the lack of any rule for what a share among nobody should be. The traceback gives you the route back from one to the other.

Five kinds of error

The first word of the last line names the kind of error, and each kind points at a different sort of mistake.

Kind What Python could not do Look for
SyntaxError read the program at all a missing colon, bracket or quote, or = where == was meant
NameError find a name a misspelling, or a name used before it has a value, or outside its function
TypeError do an operation on that type text mixed with a number, or a None from a function that returned nothing
IndexError find that position a position equal to the length, or an empty list
KeyError find that key a key that was never stored, or is spelt differently

A SyntaxError is the only one found before the program starts: nothing runs at all. Every other kind appears only when the line is reached.

scores = [70, 85, 92]
print(scores[len(scores)])

  File "your_program.py", line 2
    print(scores[len(scores)])
IndexError: list index out of range

Three items sit at positions 0, 1 and 2. len(scores) is 3, and there is no position 3.

Find the first wrong value

For a wrong answer there is no message, so you make your own evidence. The method is always the same.

  1. Decide what the program should do for one small input, by hand, before running it.
  2. Run it one line at a time, and at each step compare the values you see with the values you expected.
  3. Stop at the first value that surprises you. The bug is on that line or before it. Everything after is a consequence and can be ignored.
def average(values):
    total = 0
    for value in values:
        total = value
    return total / len(values)

print(average([4, 8, 6]))

2.0

The average of 4, 8 and 6 is 6. Stepping through, you expect total to be 4, then 12, then 18. It is 4, then 8. That is the first surprise, and it is on the line that sets total: the line replaces the total where it should add to it.

Notice what you did not do: you did not read the program hoping to spot the mistake. Reading finds some bugs. Comparing what happens with what you predicted leads you to the line, for any bug that this input brings out. It says nothing about bugs the input does not reach, and your prediction can be wrong too, so afterwards try other cases; the next two sections are about choosing them.

Say what must be true

assert states something that must be true at that point. If it is true, nothing happens and the program carries on. If it is false, the program stops there with an AssertionError.

def double(n):
    return n * 2

assert double(4) == 8
assert double(0) == 0
print("all passed")

all passed

An assert about a function is a test: a call, and the answer it should give. A handful of them under a function check it every time the program runs, in a fraction of a second, and they go on checking it after you change the function next week.

A test that fails tells you exactly which case is wrong:

def double(n):
    return n + 2

assert double(2) == 4
assert double(4) == 8

  File "your_program.py", line 5
    assert double(4) == 8
AssertionError

The first assert passes, because 2 + 2 happens to be 4. The second stops the program. One passing test proves very little, which leads to the next section.

The cases you did not picture

When you write a function you have a typical input in mind, and the function usually works for it. Bugs live in the inputs you did not picture. Four are worth trying every time:

  • empty: an empty list, an empty string, zero;
  • one item: a list of one, a single character;
  • the boundary: the value exactly at a limit, the first and last positions;
  • repeats, and all alike: equal values, every number negative, the same word twice.
def largest(values):
    best = 0
    for value in values:
        if value > best:
            best = value
    return best

assert largest([3, 9, 2]) == 9
print(largest([-5, -2, -9]))

0

The test with typical numbers passes. The case with every number below zero gives 0, a value that is not even in the list, because best started at 0. You met this function in Lists; the point here is that the ordinary test could never have found the bug. Only the awkward case reaches it.

Test first, then fix

When you have found a bug, do not fix it yet. First write an assert that fails because of it, and run the program to watch it fail. Then fix the function, and watch the same assert pass.

The order matters for two reasons. A test that fails on the broken program proves that the test can see this bug; a test written afterwards might pass for some other reason, and you would never know. And the test stays in the program, so if a later change brings the bug back, it is caught at once.

def count_above(values, limit):
    count = 0
    for value in values:
        if value > limit:
            count = count + 1
    return count

assert count_above([1, 5, 9], 4) == 2
assert count_above([3, 5, 5, 8], 5) == 1
print("all passed")

all passed

The second assert is the one that matters. It uses values equal to the limit, the boundary case, and it would have failed on a version that wrote >=.

In the lab

You will read an error report three calls deep, name the kind of error five mistakes raise, find the first wrong value in a function that runs and is wrong, decide which asserts stop a program, add the test that an ordinary case could not have been, write a failing test before a fix, and then repair six programs, three that crash and three that quietly give the wrong answer.

Your turn

Now try it yourself

Predict what a program will do, run it, and step through it line by line. The checks tell you when your own code works.

Loading the lab…

In this lab

  1. Read it from the bottom

    This program stops with an error, three calls deep. Decide where it stops and how it got there. Afterwards, step through it: the boxes of calls in progress when it stops are the traceback, drawn.

  2. Name that error

    Five small programs, each with one mistake. Before running each, name the kind of error Python will report.

  3. The first wrong value

    average(values) should return the mean of a list of numbers. It runs, and for 4, 8 and 6 it prints 2.0. The answer should be 6.0.

    Do not look for the mistake by reading. Work out by hand what total should be after each turn of the loop. Then use Step through, and stop at the first step where total is not what you expected. Fix that line.

  4. Which asserts fail?

    clamp(n, low, high) is meant to keep n within a range: below low it gives low, above high it gives high, and otherwise n itself. Four asserts test it. Decide what happens when the program runs.

  5. The case nobody pictured

    largest(values) passes all three of its tests. It is still wrong.

    1. Under the comment, add an assert for a list whose numbers are all below zero. Run the program and watch that assert fail.
    2. Add one more: for an empty list, largest should return None.
    3. Then fix the function so that every assert passes.

    Keep your asserts in the program. The check tries them on the function as it was, and they must stop it.

  6. Test first, once

    count_above(values, limit) should count the values greater than limit. It also counts values equal to the limit. Its one test does not notice.

    Do this in order.

    1. Under the comment, write an assert that the function fails as it is now. Choose values that include the limit itself.
    2. Run the program and watch your assert stop it.
    3. Only then fix the function, and run again to watch the assert pass.

    Keep your assert in the program.

  7. Repair shop: crashes

    Three small functions, and three calls that use them. The program stops at the first call. Read the message, find the mistake, and make the smallest change that fixes it. Run again: the next call will stop it. Repair all three.

    What each should do:

    • initials(first, last) returns the first letter of each name, joined. If a name is an empty string, it contributes nothing: for "Ada" and "" the answer is A.
    • price_of(prices, item) returns the item's price, or 0 for an item that is not in the dictionary.
    • last_score(scores) returns the last score in the list.

    When all three are repaired the program prints A, then 0, then 92.

  8. Repair shop: wrong answers

    Three functions that run without any error and give wrong answers. For each one, in this order: write an assert under it that fails, run the program and watch it fail, then fix the function.

    What each should do:

    • is_even(n) returns True when n is even, and False when it is odd.
    • total_price(prices) returns the sum of a list of prices, and 0 for an empty list.
    • shortest(words) returns the shortest word in a list that has at least one word. If two are equally short, the first of them.

    Keep all your asserts in the program. The checks try them on each function as it was.

  9. Tests that catch

    middle(a, b, c) returns the middle of three numbers: the one that would be second if they were put in order. If two of them are equal, the answer is still the second in order, so the middle of 1, 1 and 2 is 1. The version in the editor is correct, and you do not need to read it.

    Your job is to write the tests. Under the function, write asserts that pin down what middle must do. The check runs your asserts against the correct version, which they must pass, and against four wrong versions, each of which they must stop.

    You are not shown the wrong versions. Think about the cases a careless version would get wrong: the middle in each of the three positions, and numbers that repeat.

  10. Three bugs

    This is the equipment checkout from Dictionaries and Sets, with three separate bugs. What it should do:

    • go through the requests in order; each is a pair of an item and a number wanted;
    • a request for an item that is not stocked at all is ignored;
    • if there are at least as many as wanted, lend them; otherwise add the item's name to the refusals;
    • return a pair: a new dictionary of what is left, and the list of refusals. The stock it was given must be unchanged.

    For each bug: write an assert under the function that fails because of it, watch it fail, then fix it. Keep all your asserts. The checks try them on three versions of the function, each with one of the bugs.

  11. Edge cases for a streak

    This is a correct longest_streak(steps, goal) from Lists: the longest run of days in a row on which the step count reached the goal. You do not need to change it.

    Write the asserts a careless version would fail. There are three such versions, and each gets one kind of case wrong:

    • no days at all;
    • days, but none that reaches the goal;
    • the longest streak running to the very last day.

    Your asserts must pass on the correct version and stop all three.

  12. Bug diary

    Take a program of your own from an earlier module that works. Break it on purpose, twice, in two different ways: once so that it crashes, and once so that it runs and gives a wrong answer. Paste the broken versions here one at a time and run them.

    For each, write as comments: the message or the wrong output you saw; the line where the mistake really is; and how you would have found it if you had not planted it yourself.

    There are no checks. Go through the list below when you are done.

Was this lesson useful?Your feedback helps us improve the course.