Testing and Debugging

A program that compiles is not yet a program that is right. A mistake in a program is called a bug. Testing is how you find out whether a program has bugs. Debugging is finding the line that is wrong and repairing it.

Until now the lab's checks did the testing for you. This lesson shows how to do it yourself.

Three ways to be wrong

A program can be wrong in three ways. They differ in who notices.

The first way you know from module 1: a compile error. The compiler does not accept the program, and nothing runs.

#include <iostream>

int main() {
    int total = 7 + 8;
    std::cout << "Total: " << totl << "\n";
}

The name totl is misspelt. The compiler reports the line, and you repair it. This is the easiest kind of bug, because the compiler finds it for you.

The second way is a run-time error: the program compiles and starts, and then stops before it has finished its work. This program shares sweets among some children.

#include <iostream>

int main() {
    int sweets = 12;
    int children = 0;
    std::cin >> children;
    std::cout << "Each child gets " << sweets / children << "\n";
    std::cout << "Done\n";
}

Each child gets 4
Done

With the input 3 it works. With the input 0 it cannot: 12 divided by 0 has no answer. C++ does not say what a program does then. In the lab the program is stopped at the line with the division, and Done is never printed.

Notice that the compiler had nothing to complain about. sweets / children is correct C++. Whether it works depends on a value that exists only while the program runs.

The third way is a wrong answer: the program compiles, runs to its end, and what it prints is not true.

#include <iostream>

int main() {
    int first = 7;
    int second = 8;
    double average = (first + second) / 2;
    std::cout << "Average: " << average << "\n";
}

Average: 7

The average of 7 and 8 is 7.5. Both sides of the / are int, so the fraction was dropped before the result reached average. Nothing stopped and nothing was reported.

Kind	What you see	Who notices
compile error	a message from the compiler; nothing runs	the compiler
run-time error	the program starts, then stops early	the lab, at the moment it happens
wrong answer	a normal run, with false output	only you

The third kind is the dangerous one. No tool will tell you that 7 is the wrong average. You have to know what the right answer is, and compare.

Decide the answer first

Here is a function with a rule that is easy to state: a ticket costs 5 for a child under 12 and 9 for anyone else.

To test it, choose an input, and work out from the rule what the function should return. Do this before you run anything. An input together with the output you expect is a test case.

Input: age	Expected price	Why
8	5	8 is under 12
30	9	30 is not under 12

Then run the function with each input and compare.

#include <iostream>

// Returns the price of a ticket: 5 for a child under 12, 9 for anyone else.
int ticket_price(int age) {
    if (age < 12) {
        return 5;
    }
    return 9;
}

int main() {
    std::cout << ticket_price(8) << "\n";
    std::cout << ticket_price(30) << "\n";
}

5
9

Both lines match the table, so both test cases pass. A test case whose output differs from what you expected fails.

The order matters. If you run the program first and then ask yourself whether 5 looks right, you will nearly always say yes. An expected answer copied from the program's output can never disagree with the program, so it tests nothing. The expected answer must come from the rule.

Where programs break

The next version of ticket_price has a bug. Read its condition.

#include <iostream>

// Returns the price of a ticket: 5 for a child under 12, 9 for anyone else.
int ticket_price(int age) {
    if (age <= 12) {
        return 5;
    }
    return 9;
}

int main() {
    std::cout << ticket_price(8) << "\n";
    std::cout << ticket_price(30) << "\n";
    std::cout << ticket_price(11) << "\n";
    std::cout << ticket_price(12) << "\n";
    std::cout << ticket_price(13) << "\n";
}

The two test cases from the last section, 8 and 30, still pass. So do 11 and 13. Only the input 12 gives a wrong price: the rule says 9, and the function returns 5.

Of all the ages there are, exactly one exposes this bug, and it is the one where the rule changes from one price to the other. An input at such a place is an edge case. Bugs collect at the edges, because that is where < and <=, or 0 and 1, make a difference. In the middle of a range, a right version and a wrong version usually agree.

When you choose test cases, look for the edges first.

Each boundary in the rule. For "under 12", test 11, 12 and 13: one below, on, and one above.
Zero and one. A loop that runs zero times or once, a count of 0, an empty input.
The smallest and largest inputs the function allows.
Equal values, when the function compares its inputs with each other.
Every listed case. If a rule names several special values, test each of them, not only the first.

Add one ordinary input from the middle of each range as well. Two or three of those are enough.

assert

Comparing printed numbers with a table by eye is slow, and you will stop doing it. C++ can do the comparing for you.

#include <cassert>
#include <iostream>

// Returns the price of a ticket: 5 for a child under 12, 9 for anyone else.
int ticket_price(int age) {
    if (age < 12) {
        return 5;
    }
    return 9;
}

int main() {
    assert(ticket_price(11) == 5);
    assert(ticket_price(12) == 9);
    std::cout << "Both held\n";
}

Both held

assert(condition); states that the condition is true at this line. It comes from the header <cassert>.

If the condition is true, nothing happens. Nothing is printed, and the program carries on with the next statement.
If the condition is false, the program stops at that line. No later statement runs.

Each assert here is one test case: the input is inside the call, and the expected output is on the right of ==.

Now the same main with the buggy function from the last section.

#include <cassert>
#include <iostream>

// Returns the price of a ticket: 5 for a child under 12, 9 for anyone else.
int ticket_price(int age) {
    if (age <= 12) {
        return 5;
    }
    return 9;
}

int main() {
    assert(ticket_price(11) == 5);
    assert(ticket_price(12) == 9);
    std::cout << "Both held\n";
}

The first assert is true, and nothing happens. The second is false: ticket_price(12) is 5, not 9. The program stops there, and Both held is never printed. The lab's status line then shows a message that names the line of the failed assert, here line 14, and repeats its condition.

That is a run-time error, and one you asked for. An assert turns a wrong answer, which nobody notices, into a stop at a known line, which nobody can miss.

A program in which every assert is true prints nothing extra. Silence means that everything you stated was true.

A function that tests a function

Test cases belong together, in a function of their own. A test function calls one function with several inputs and asserts each result. By custom its name is test_ followed by the name of the function it tests.

#include <cassert>
#include <iostream>

// Returns the price of a ticket: 5 for a child under 12, 9 for anyone else.
int ticket_price(int age) {
    if (age < 12) {
        return 5;
    }
    return 9;
}

void test_ticket_price() {
    assert(ticket_price(0) == 5);     // the smallest age
    assert(ticket_price(8) == 5);     // an ordinary child
    assert(ticket_price(11) == 5);    // one below the boundary
    assert(ticket_price(12) == 9);    // on the boundary
    assert(ticket_price(13) == 9);    // one above the boundary
    assert(ticket_price(30) == 9);    // an ordinary adult
}

int main() {
    test_ticket_price();
    std::cout << "All tests passed\n";
}

All tests passed

main calls the test function first. If every assert is true, the call returns and main prints its line. If one is false, the program stops inside test_ticket_price, and the message names the line of the test case that failed.

Three things make a test function worth keeping.

It can be run again. After every change to ticket_price, one run tells you whether the old cases still pass.
It says what the function should do, not how. Nothing in test_ticket_price depends on how ticket_price is written. The same six lines would test any version of it.
A good one tells versions apart. These tests pass with the correct function and stop with the <= version, because of the case for 12. A test function with only the cases 8 and 30 would pass with both, and so would tell you nothing about the boundary.

When a test fails, there are two places to look: the function, and the test. Check first that the expected value in the assert really follows from the rule.

Two bugs you cannot see

Two bugs compile, run, and look correct on the screen. Both are about which statements an if or a loop controls.

The first is a semicolon in the wrong place.

#include <iostream>

int main() {
    int temperature = 20;
    if (temperature < 0); {
        std::cout << "Freezing\n";
    }
    for (int i = 0; i < 3; ++i); {
        std::cout << "row\n";
    }
}

Freezing
row

It is not freezing at 20 degrees, and row should appear three times.

Look at the ; straight after if (temperature < 0). A semicolon on its own is a complete statement that does nothing. So the if controls that empty statement, and the if ends there. The braces below are then an ordinary group of statements, and they run every time.

The for line has the same semicolon. The loop does run three times, and each time it runs the empty statement. Then the loop is over, and the braces below run once.

The rule: no semicolon between the ) of an if, for or while and its {.

The second bug is braces that are not there.

#include <iostream>

int main() {
    int coins = 3;
    if (coins >= 10)
        std::cout << "Enough for a ticket\n";
        std::cout << "Printing the ticket\n";
    std::cout << "Goodbye\n";
}

Printing the ticket
Goodbye

Three coins are not enough, and still the ticket is printed.

C++ allows an if or a loop without braces. Then it controls exactly one statement: the next one. Here that is the first std::cout line. The second std::cout line is indented as though it belonged to the if, and it does not. The compiler ignores indentation completely.

The rule: always write the braces, even around one statement. Every program in this course does.

Looking inside a running program

A test tells you that a function is wrong. It does not tell you which line. To find the line you need to see the values the variables hold while the program runs. The lab gives you two tools for that.

The first is watch. The call watch("name", value); records one value under a name. It comes from scimigo.h, like the drawing calls.

#include "scimigo.h"
#include <iostream>
using namespace scimigo;

int main() {
    int power = 1;
    for (int i = 0; i < 5; ++i) {
        power = power * 2;
        watch("power", power);
    }
    std::cout << power << "\n";
}

The watch line runs five times, once on each turn of the loop. The lab's Watch panel then lists five values for power, in the order they were recorded: 2, 4, 8, 16 and 32. The output panel is unchanged. watch prints nothing.

Use watch when you suspect one variable. Before you run, write down the values you expect it to take. Then compare your list with the panel. The first value that differs tells you on which turn things went wrong.

The second tool is the Step through button. It runs the program one statement at a time. After each statement it shows the statement that will run next, and the value of every variable at that moment.

Use stepping when you do not know which variable to suspect. Work like this.

Before each step, say what the next statement will change, and to what value.
Press the button and compare.
Stop at the first statement where what you see differs from what you said. Either that statement is the bug, or the statement did not run when you thought it would, and the condition that guards it is the bug.

Both tools are for looking. They do not repair anything, and a watch line can be deleted when the bug is found.

What a function promises

Every test case needs an expected answer, and the expected answer comes from knowing exactly what the function is supposed to do. The place to write that down is a comment above the function. Such a comment is called the function's contract. It has two parts.

Needs: what must be true of the values the caller gives.
Returns: what the function gives back, if the needs are met.

#include <cassert>
#include <iostream>

// Needs: whole is greater than 0, and part is 0 or more.
// Returns: part as a percentage of whole, with the fraction dropped.
int percent(int part, int whole) {
    assert(whole > 0);
    assert(part >= 0);
    return part * 100 / whole;
}

int main() {
    std::cout << percent(1, 4) << "\n";
    std::cout << percent(2, 3) << "\n";
}

25
66

A contract divides the work between two people, even when both are you.

The caller must meet the needs. A call such as percent(1, 0) breaks the contract, and that is a bug in the caller, not in percent. The function has promised nothing for it.
The function must keep the promise for every input that meets the needs. percent(2, 3) must be 66, because the contract says the fraction is dropped.

The two assert lines at the top of percent check the needs. If a caller breaks the contract, the program stops there, at once, instead of carrying on with a meaningless number.

The contract also tells you what to test. Each sentence of "Returns" gives you test cases with their expected answers. "Needs" tells you which inputs are not test cases at all: there is no right answer for percent(1, 0), so there is nothing to assert.

A good contract passes this test: someone who has never seen the body of the function could write the body from the comment alone.

Your turn

Now try it yourself

Predict what a program will do, run it, and step through it line by line. The checks tell you when your own code works.

Loading the lab…

In this lab

Three kinds of wrong

Three programs follow. Each is wrong in a different way. Read all three before you answer.

Program A should print a total.

#include <iostream>

int main() {
    int total = 7 + 8;
    std::cout << "Total: " << totl << "\n";
}

Program B is given the input 0.

#include <iostream>

int main() {
    int pages = 120;
    int days = 0;
    std::cin >> days;
    std::cout << "Pages per day: " << pages / days << "\n";
    std::cout << "Done\n";
}

Program C should print the average of 7 and 8. It is the program you can run here.

Decide before you run
Suppose someone has written is_leap_year(year), which returns true or false. You have not seen its body. You have only the rule:

A year is a leap year if it is divisible by 4. Except: a year divisible by 100 is not a leap year. Except: a year divisible by 400 is a leap year after all.

Before the function is run even once, decide what it should return. There is nothing to run in this step.
Where would it break?
clamp(value, low, high) limits a value to a range. If value is below low it returns low. If value is above high it returns high. Otherwise it returns value unchanged. Both ends belong to the range.

You are going to test it with low = 0 and high = 100. Here are eight candidates for value:
```
-50    -1    0    1    50    99    100    101
```
The program below has a clamp with a bug in it. Its main tries three ordinary values, and all three come out right.

The line that stops

This program has three assert statements. Read it and decide what happens. Its line numbers are shown, because the answer mentions them.

 1  #include <cassert>
 2  #include <iostream>
 3
 4  int main() {
 5      int n = 5;
 6      assert(n > 0);
 7      assert(n > 10);
 8      assert(n < 100);
 9      std::cout << "end\n";
10  }

There is nothing to run in this step.

A function that tests a function
max_of_three should return the largest of its three values. test_max_of_three has two test cases, and both pass. Press Run and see All tests passed.

Two passing cases are not much. In both of them the three values are different. Add at least four cases to test_max_of_three in which values are equal:
- the largest value is in the first and second places,
- the largest value is in the first and third places,
- the largest value is in the second and third places,
- all three values are the same.
Work out each expected answer from the first sentence above, not from the code. Run again. One of your new cases stops the program.

Then repair max_of_three, so that every case passes. Do not change or remove a test case to make it pass.
The semicolon that ends the if
Look closely at the end of the if line and the end of the for line. Decide what this program prints, then run it.
Which statements belong?
This if has no braces. The two lines below it are indented the same way. stock is 0, so the condition is false.
Watch the total
Three students each took four tests. The program reads the twelve scores, one student at a time, and should print each student's average:
```
Student 1: 85
Student 2: 65
Student 3: 95
```
Press Run. The first line is right and the others are not.
1. Add this line straight after the statement that adds a score to total:
```
watch("total", total);
```
2. Run again and read the Watch panel. It lists twelve values of total, one for each score. Before you look, say what the fifth value should be.
3. One statement in this program is in the wrong place. Move it. Change nothing else.
Step to the bug
digit_count returns how many digits a whole number has. For 507 it returns 3. For 0 it returns 0, and that is wrong: 0 is written with one digit.
1. Before you step, say which statement should make count become 1 during the call digit_count(0).
2. Press Step through and step until you are inside that call. Watch n and count after each step.
3. Stop at the place where count should have changed and did not. Which statement did not run, and what kept it from running?
Then repair digit_count so that it returns 1 for 0 and is still right for every other number its contract allows.

What does it promise?

Here is a function's first line with its contract. You do not see the body.

// Needs: low is not greater than high.
// Returns: value limited to the range low to high. A value below low
//          gives low, a value above high gives high, and any other
//          value is returned unchanged.
int clamp(int value, int low, int high)

Four calls, each written as clamp(value, low, high):

clamp(5, 0, 10)      clamp(-3, 0, 10)      clamp(5, 10, 0)      clamp(7, 7, 7)

Tests that catch the bugs
days_in_month(month, leap_year) returns the number of days in a month. month is from 1 to 12. leap_year says whether the year is a leap year, which matters for February only: 29 days in a leap year, 28 otherwise.

Write its test function, test_days_in_month. One test case is there as a model.

Check runs your test function five times. The first time, days_in_month is the correct version in the starter, and every one of your assert lines must be true. The other four times, a different version is swapped in. Each of those four has one bug. Your test function must stop on every one of them.

You are not told what the four bugs are. Think about where a function like this could go wrong, and write a test case for each place.

One line of the starter is not yours to change: the line that begins int (*days_in_month). It is how the checks swap versions, and it uses C++ from a much later module. In your tests, call days_in_month(...) as you would call any function, and write each expected number yourself.
Three bugs
This program looks at the whole numbers from 1 to limit and reports on those that are multiples of 3. With limit at 10 the multiples are 3, 6 and 9, so it should print:
```
Count: 3
Total: 18
Average: 6
```
It has three bugs, one of each kind from the lesson. You will meet them in this order: first it does not compile; then it starts and stops early; then it runs to the end and prints a wrong number.

Find and repair all three. The repairs are small: no line needs to be rewritten. Use watch or Step through where they help.

At the end of each line you repair, add a comment that names the kind of bug as you met it. Use exactly these words:
```
// compile error
// run-time error
// wrong answer
```
For a repair that adds braces, put the comment on the line with the opening brace. Leave the assert in average where it is: it is not one of the bugs.
Write it with its tests
The median of three values is the one in the middle when they are put in order. The median of 7, 2 and 5 is 5. The median of 4, 9 and 4 is 4.

Write three things.
1. The function. Complete my_median_of_three so that it returns the median of a, b and c. Any three whole numbers may be given, in any order, and some may be equal.
2. Its contract. Replace the comment above the function with a contract comment: what it needs, and what it returns. If it needs nothing, say so.
3. Its tests. Write test_median_of_three with at least six test cases. In the tests, call the function as median_of_three(...).
As in "Tests that catch the bugs", one line is not yours to change: the line that begins int (*median_of_three). It makes median_of_three(...) call your function, and lets the checks swap in other versions.

Check does three things. It calls your function with values of its own. It runs your tests against your function, and every case must pass. Then it runs your tests against four wrong versions, and your tests must stop on each.
Tests for is_prime
A prime is a whole number greater than 1 that can be divided, with no remainder, only by 1 and by itself. is_prime(n) returns whether n is prime. Any int may be given.

Write test_is_prime. Check runs it against the correct is_prime in the starter, where every case must pass, and then against five wrong versions. Your tests must stop on each of the five.

As before, leave the line that begins bool (*is_prime) as it is, call is_prime(...) in your tests, and decide each expected answer yourself.

A case for a number that is prime is assert(is_prime(7));. A case for one that is not is assert(!is_prime(8));.
Five one-line bugs
Five small functions, each with its contract. Each has exactly one mistake. main calls each of them once, and when all five are right it prints:
```
double_it(4) = 8
absolute(5) = 5
sum_to(4) = 10
is_zero(0) = 1
halvings(8) = 3
```
Repair each function with the smallest change you can. Four of the repairs change one line. One of them needs a pair of braces.

The program does not compile at first. Start with the compiler's message.
A table of tests
In module 4 you classified a triangle from the lengths of its three sides. Here that is a function, triangle_kind(a, b, c). It returns one of four numbers, which have names: not_a_triangle, equilateral, isosceles and scalene. The names are constants declared above every function, so that every function can use them. Its contract is in the starter. Read it closely: every sentence in it is a source of test cases.

First make a table on paper with ten rows: three side lengths, and the kind you expect. Then write each row as one assert in test_triangle_kind, for example:
```
assert(triangle_kind(3, 4, 5) == scalene);
```
Check runs your table against the correct function, where every row must pass, and against three wrong ones. Your table must stop on each of them.

Leave the line that begins int (*triangle_kind) as it is.
Promises
Three functions you met in module 6 are below, with their bodies and without their contracts.

Write a contract comment above each: a Needs line and a Returns line. draw_house returns nothing, so its second line says what it draws instead.

Read each body to find out what the function really does at its edges, and what it would do with values it was not meant for. Then decide which of those values the contract should rule out.

Nothing checks this one. When you have written the three comments, go through the list below.
A bug diary
For the next three bugs you meet in this course, keep a record. You can keep it on paper, or as comments in the program below.

For each bug write down four things: which of the three kinds it was, how you noticed it, how you found the line, and roughly how long that took.

Nothing checks this one. After the third bug, go through the list below.

Testing and Debugging

Three ways to be wrong

Decide the answer first

Where programs break

assert

A function that tests a function

Two bugs you cannot see

Looking inside a running program

What a function promises

Now try it yourself

In this lab

Three kinds of wrong

Decide before you run

Where would it break?

The line that stops

A function that tests a function

The semicolon that ends the if

Which statements belong?

Watch the total

Step to the bug

What does it promise?

Tests that catch the bugs

Three bugs

Write it with its tests

Tests for is_prime

Five one-line bugs

A table of tests

Promises

A bug diary