Build with Python

Pythonic Data Work

13 of 24LabBuild a GameRecursion
On this page

Pythonic Data Work

You can already do everything in this module. You can number the items of a list, walk two lists together, build a new list from an old one, add up, find the largest, and put things in order, because you have written the loops for all of them. Those loops are so common that Python has a short form for each. This module teaches the short forms, always beside the loop you already know, so that you can read them in other people's code and choose between the two in your own.

Before this, you can: loop over a list by item or by position and build a new list with append; write a function that returns a value; keep pairs in a list and unpack them.

By the end you will be able to get the position and the item together with enumerate, walk two lists side by side with zip, build and filter a list in one line, use sum, min and max, sort with sorted, say what to sort by with key=, and tell when a plain loop is still the better choice.

Before you read on, decide what this prints:

scores = [70, 85, 60]
ordered = sorted(scores)
print(scores)
print(ordered)

[70, 85, 60]
[60, 70, 85]

sorted did not put scores in order. It made a new list, in order, and left the original as it was. Every short form in this module behaves that way: it gives you something new and changes nothing you already had. That is the first reason to like them.

The position and the item together

When a loop needs both where an item is and what it is, you have been writing this:

runners = ["Ana", "Ben", "Chen"]
for i in range(len(runners)):
    print(i, runners[i])

0 Ana
1 Ben
2 Chen

enumerate hands you both at once, as a pair that the for line unpacks:

runners = ["Ana", "Ben", "Chen"]
for i, name in enumerate(runners):
    print(i, name)

0 Ana
1 Ben
2 Chen

There is no range, no len, and no runners[i] to get wrong. If the counting should start somewhere else, say so:

runners = ["Ana", "Ben", "Chen"]
for place, name in enumerate(runners, start=1):
    print(place, name)

1 Ana
2 Ben
3 Chen

Only the numbers change. start=1 does not skip the first runner.

Two lists side by side

Two lists that belong together, a name and a score at each position, have also meant an index loop: names[i] and scores[i]. zip walks both at once and hands you one item from each:

names = ["Ana", "Ben", "Chen"]
scores = [70, 85, 60]
for name, score in zip(names, scores):
    print(name, score)

Ana 70
Ben 85
Chen 60

If the lists are not the same length, zip stops when the shorter one runs out, and says nothing about it:

names = ["Ana", "Ben", "Chen"]
scores = [70, 85]
for name, score in zip(names, scores):
    print(name, score)

Ana 70
Ben 85

Chen has vanished without an error. When two lists are meant to match, that silence can hide a bug, so it is worth an assert len(names) == len(scores) first.

zip gives its pairs one at a time, to a loop. To keep them, make a list of them: list(zip(names, scores)) is a list of pairs.

A new list in one line

Building a list from another list has been three lines: an empty list, a loop, an append.

prices = [4, 10, 6]
doubled = []
for price in prices:
    doubled.append(price * 2)
print(doubled)

[8, 20, 12]

A list comprehension says the same thing inside one pair of square brackets:

prices = [4, 10, 6]
doubled = [price * 2 for price in prices]
print(doubled)
print(prices)

[8, 20, 12]
[4, 10, 6]

Read it from the middle: for price in prices is the loop you know, and what stands before it, price * 2, is what goes into the new list on each turn. The brackets make the list. prices is untouched.

The expression at the front can be anything that has a value: len(word), name.upper(), a call to a function of your own.

Keeping only some

A loop that keeps some items and drops others has an if around its append. A comprehension takes the same if, at the end:

scores = [70, 85, 60, 92]
passed = [score for score in scores if score >= 70]
print(passed)

[70, 85, 92]

The order of the three parts is fixed: what to keep, the loop, the test. When nothing passes the test, the result is an empty list, which is a perfectly good answer:

scores = [70, 85, 60, 92]
print([score for score in scores if score > 100])

[]

The front and the end do different jobs. [score + 15 for score in scores if score < 80] tests each score as it was, and then adds 15 to the ones it keeps: 70 passes the test and goes into the list as 85.

Total, smallest, largest

In Lists you wrote a running total and a best-so-far by hand. Python has them:

scores = [70, 85, 60, 92]
print(sum(scores), min(scores), max(scores))
print(sum(scores) / len(scores))

307 60 92
76.75

They take any list, including one a comprehension has just made:

scores = [70, 85, 60, 92]
print(sum([1 for score in scores if score >= 70]))

3

That counts the passes: a 1 for each, added up.

You wrote largest yourself in Lists and had to decide what it should do for an empty list. Python's authors had to decide too, and they chose to stop:

print(max([]))

  File "your_program.py", line 1
    print(max([]))
ValueError: max() iterable argument is empty

sum([]) is 0, because adding nothing up gives nothing. But no number is the largest of none, so max and min refuse. Check for the empty list first if it can happen.

In order

sorted gives a new list with the same items in order, smallest first:

scores = [70, 85, 60, 92]
print(sorted(scores))
print(sorted(scores, reverse=True))
print(scores)

[60, 70, 85, 92]
[92, 85, 70, 60]
[70, 85, 60, 92]

Strings are put in alphabetical order, with one surprise: each of the capital letters A to Z comes before every one of the small letters a to z. (Letters with accents, and other alphabets, have places of their own, after both.)

print(sorted(["pear", "Apple", "fig", "Zebra"]))

['Apple', 'Zebra', 'fig', 'pear']

Lists also have a method, sort, which is a different thing. It puts that list in order, in place, and gives back None:

scores = [70, 85, 60]
result = scores.sort()
print(scores)
print(result)

[60, 70, 85]
None

So scores = scores.sort() loses you the list: it sorts it, and then makes the name scores refer to None. The sorted list still exists only if some other name refers to it. And sorted(scores) on a line by itself does nothing useful, because the new list is thrown away. Both mistakes are easy to make and neither raises an error.

Sorted by what?

Put four words in order and you get the alphabet. To order them some other way, give sorted a key: a function that it calls on each item, to get the value to sort by.

words = ["pear", "fig", "banana", "kiwi"]
print(sorted(words))
print(sorted(words, key=len))

['banana', 'fig', 'kiwi', 'pear']
['fig', 'pear', 'kiwi', 'banana']

key=len hands over the function len itself, without calling it, as you handed clicked to on_click. sorted calls it once for each word and orders the words by the answers: 3, 4, 4, 6. Two words with the same key, pear and kiwi, stay in the order they were in.

The key can be a function of your own. To order pairs by their second item:

def score_of(pair):
    return pair[1]

results = [("Ana", 70), ("Ben", 85), ("Chen", 60)]
print(sorted(results, key=score_of, reverse=True))

[('Ben', 85), ('Ana', 70), ('Chen', 60)]

A function that small, used in one place, can be written where it is used. That is what lambda is for:

results = [("Ana", 70), ("Ben", 85), ("Chen", 60)]
print(sorted(results, key=lambda pair: pair[1], reverse=True))

[('Ben', 85), ('Ana', 70), ('Chen', 60)]

lambda pair: pair[1] is the function score_of without a name: it takes pair and gives back pair[1]. A lambda holds one expression and nothing else. Anything longer deserves a def and a name.

max and min take a key too. max(results, key=score_of) is the pair with the highest score, and the first such pair if two are level.

When the loop is still the better choice

The short forms are for one job: making a value from a list. They are the wrong tool when the loop is there to do something.

names = ["Ana", "Ben"]
for name in names:
    print("Hello", name)

Hello Ana
Hello Ben

Writing that as [print("Hello", name) for name in names] works, and builds a list of two Nones that nobody wants. Drawing, printing, and changing a list in place are things to do, so they stay loops.

A loop is also better when a comprehension stops being readable at a glance: two fors, a long test, or a step you would want to print while finding a bug. You cannot step through the inside of a comprehension as you can step through a loop. When in doubt, write the loop, get it right, and shorten it afterwards if it reads better short.

In the lab

You will predict what enumerate, zip, comprehensions, sum, sorted and key= give, turn a hand-kept counter into enumerate and a build-a-list loop into a comprehension, write a leaderboard that ranks names by score, find the most improved player from before and after scores, pick out finalists in alphabetical order, shorten three loops to one line each, and repair a program that sorts and gets nothing for it.

Your turn

Now try it yourself

Predict what a program will do, run it, and step through it line by line. The checks tell you when your own code works.

Loading the lab…

In this lab

  1. Position and item

    Two loops over the same list, and a third that counts from somewhere else. Decide what each prints.

  2. Numbered list

    This program draws a numbered to-do list. It keeps the number in a variable of its own, number, and adds 1 to it by hand on every turn.

    Draw exactly the same picture with enumerate, and with no number variable to look after. The checks try other lists.

  3. Side by side

    zip walks two lists at once. Decide what each program prints.

  4. One line

    A loop that builds a list, and the comprehension that says the same thing. Decide what each prints.

  5. Keeping only some

    A comprehension with an if at the end keeps only the items that pass the test. Decide what each one gives.

  6. Bonus round

    In the bonus round only even scores count, and each of them is squared. bonus(scores) builds that list with a loop, an if and an append.

    Rewrite the body of bonus as one comprehension that returns the same list. For [3, 4, 7, 10] it returns a list holding 16 and 100.

  7. Total, smallest, largest

    sum, min and max do the loops you wrote by hand in Lists. Decide what each program does.

  8. In order

    sorted and sort both put things in order, and they are not the same. Decide what each program prints.

  9. Sorted by what?

    A key is a function that sorted calls on each item, to get the value to sort by. Decide what each program prints.

  10. Leaderboard

    A game keeps its players' names in one list and their scores in another, in the same order.

    Write ranked(names, scores). It returns a list of (name, score) pairs, highest score first. Players level on score keep the order they had in the lists. Neither list is changed.

    Then draw the board from what ranked returns. For the player in place n, counting from 1:

    • the place, a full stop, a space and the name, written at x = 40, y = 50 × n + 20;
    • a "gold" bar starting at x = 200, y = 50 × n, 30 tall and 3 wide for every point.

  11. Most improved

    A class sat a test twice. names, before and after are three lists in the same order.

    Write most_improved(names, before, after). It returns the name of the pupil whose score rose the most. If two pupils rose by the same amount, it returns the one who comes first in the lists. The lists are never empty.

    Then draw the chart. For pupil number i, counting from 0:

    • a "lightgray" bar for the first score at x = 40 + 130 × i, 40 wide, 3 tall for every point, with its bottom edge at y = 340;
    • a bar for the second score beside it, at x = 85 + 130 × i, the same width and scale. It is "seagreen" for the most improved pupil and "steelblue" for the others;
    • the name at x = 40 + 130 × i, y = 365.

  12. Finalists

    A competition keeps one record for each entrant: a list holding the name and the score, such as ["Ana", 72].

    Write finalists(records, pass_mark). It returns the names of the entrants whose score is at or above the pass mark, in alphabetical order. The list of records is not changed.

    The program then prints the finalists for the records in the starter, with a pass mark of 70.

  13. Three loops, three lines

    Three functions that work, each written with a loop. Rewrite the body of each as a single return line that gives the same answer for every input.

    • lengths(words) returns the length of each word.
    • total_over(amounts, limit) returns the total of the amounts that are greater than limit.
    • differences(first, second) returns, for each position, the item of second minus the item of first.

    When you are done the body of each function is one return line, and there is no for statement left in the program.

  14. Sorted, and nothing changed

    This program is meant to print a scoreboard twice: first the scores from lowest to highest, then the players' names from the highest score to the lowest. scores and players must stay as they are, because the rest of a larger program uses them.

    Run it. The first line is not in order, and the second is None.

    Fix the two lines that are wrong. Then, in a comment above each, say in a few words why it failed.

  15. Loop or one line?

    Four small jobs on the list in the starter. Write each one in the form you think reads best: a loop, or one of this module's short forms. There is no single right answer for most of them, but you should be able to say why you chose as you did.

    1. Make a list of the temperatures in Fahrenheit: each one times 9, divided by 5, plus 32.
    2. Print each day's number, counting from 1, and its temperature, one day to a line.
    3. Make a list of the temperatures that are below freezing (below 0).
    4. Print the warmest temperature, and on which day number it fell.

    There are no checks. Go through the list below when you are done.

Was this lesson useful?Your feedback helps us improve the course.