At EverQuote, we emphasize Heart as a core value. We use biweekly tech talks as a way to come together across teams and learn the intricacies of our tools. Here are some of the quirky things we learned about Python--hope you enjoy as much as we did!

Other folks have gone over some common gotchas, and we hope to build on that in this post.

1. Building a list of lists

Imagine we're building a blackjack engine, and we need to initialize a data structure that tracks multiple decks of cards. We decide on a two-dimensional list. Let's initialize this list by taking advantage of

>>> decks = [[]] * 5

Now suppose we try to use this initialized structure and add a single card to the first deck.

>>> decks[0].append("queen of hearts")

>>> print(decks)

You might expect this to be:

[["queen of hearts"],[],[],[],[]]

But what actually happens is:

[["queen of hearts"], ["queen of hearts"], ["queen of hearts"], ["queen of hearts"], ["queen of hearts"]]

That's not what we want! It looks like the card got added to all the decks. Why is that? Well, turns out this is equivalent to:

decks = []

deck = []

for _ in range(5):

decks.append(deck)

But what we actually wanted was:

decks = []

for _ in range(5):

deck = []

decks.append(deck)

The "one obvious way" of doing it:

>>> decks = [[] for _ in range(5)]

By using a list comprehension, we ensure that `[]` is evaluated separately for each iteration of `range(5)`, creating five separate list objects. `[[]] * 5` looks like it should accomplish the same thing, but it actually reuses the same inner list for all five elements of the outer list.

>>> decks[0].append("queen of hearts")

>>> print(decks)

[['queen of hearts'], [], [], [], []]

2. Mutable default arguments

Another place where list object literals might bite you is default arguments for functions and methods. Suppose we want the ability to add cards to a deck, creating a new deck if needed.

def add_card(card, deck=[]):

deck.append(card)

return deck

deck1 = add_card("ace of spades")

print(deck1)

deck2 = add_card("two of spades")

print(deck2)

You might expect the output to be:

["ace of spades"]

["two of spades"]

But what we actually get is:

["ace of spades"]

["ace of spades", "two of spades"]

The issue here is that deck=[] creates a persistent object identity that is bound as the default argument. The idiomatic way to handle this would be to do the following:

def add_card(card, deck=None):
   if not deck:
       deck = []
   deck.append(card)

3. Primitive integer identity

a = 256

b = 256

print(a is b)  # true

a = 257

b = 257

print(a is b)  # false?!

This counterintuitive behavior is actually due to an implementation detail of cPython. As a performance optimization, cPython maintains a static global array of Python objects for the integers -5 to 256, inclusive. Other implementations of Python (like PyPy) will not necessarily exhibit this behavior.

In Python, the `is` operator checks that the `id` of two objects are the same. That is, `a is b` is equivalent to `id(a) == id(b)`. cPython treats `id` as the location in memory. Since integer literals for the special numbers -5 to 256 always resolve to the same underlying object,

One of the primary idiomatic use cases for the `is` operator is comparison to `None`. PEP 8 actually recommends this, specifically because the behavior of `==` can actually be overwritten by the compared operand. We'll come to understand this a little more deeply as we dive into magic methods.

4. Late binding closures

A closure is a technique that scoped name binding, allowing nested functions to inherit variables from an enclosing environment. In this example, we have a nested function `print_card` that uses the free variable `card`, which is defined in its enclosing function `print_deck`.

def print_deck():

printers = []

deck = ["ace of spades", "two of spades", "three of spades"]

for card in deck:

def print_card():

print(card)

printers.append(print_card)

for printer in printers:

printer()

So this function `print_deck` creates a list of functions called `printers`, and then simply loops through each printer function to execute them all. Let's call `print_deck`.

print_deck()

You might expect output like this:

ace of spades

two of spades

three of spades

What actually happens (annotated with some comments) is this:

three of spades  # ?

three of spades  # ?!

three of spades

Since `card` is actually defined in `print_deck`, each `printer` function has a "lazy" reference to the bound variable `card`. As `print_deck` is executed, this variable is reassigned to each element of `deck`. By the end of the loop, `deck` is now assigned the value of "three of spades", so when we finally execute all the `printer`s, that's what we get!

One approach to get our desired behavior would be to use default arguments to force an early binding of the variable. `card=card` looks like an ugly hack since we're defining a new `card` that is a snapshot of the outer `card`, and that's a totally valid perspective. People unfamiliar with late binding closures might read this code and see that expression as a no-op, even though it serves the critical role of binding `card` early.

def print_deck():

printers = []

deck = ["ace of spades", "two of spades", "three of spades"]

for card in deck:

def print_card(card=card):

print(card)

printers.append(print_card)

for printer in printers:

printer()

5. Augmented assignment with tuples


Now that we've created our list of decks for our blackjack game, we want to make sure that the decks can't be altered, i.e. ensure that our list is immutable. One way to go about this is to convert it to a tuple. To test that your list of decks is indeed immutable, we try to add another card using an augmented operator i.e. "+=".

decks = tuple([["ace of spades"], ["three of clubs"], ["queen of hearts"]])

decks[2] += ["eight of diamonds"]

You might expect this to throw an exception as tuples are immutable, and you'd be right.

Traceback (most recent call last):

File "<stdin>", line 1, in <module>

TypeError: 'tuple' object does not support item assignment

However, the interpreter will still (maybe unexpectedly) append the to the nested list. If you print the tuple you'll get:

print(decks)

["ace of spades"], ["three of clubs"], ["queen of hearts", "eight of diamonds"]]

This quirk only happens if you try to use an augmented assignment `+=` on a list within a tuple.

If you run the disassembler on that line of code, dis.dis(str), you will see that STORE_SUBSCRwhich evaluates TOS1[TOS] = TOS2—is run before that particular value is returned. This line would fail if TOS1 were a tuple. Since we've already mutated the list beforehand with INPLACE_ADD, that list will stay mutated, and the exception will still be raised.

Example of using the disassembler:

from dis import dis

t = (1,[2,3])
dis("t[1] += [1]")

1           0 LOAD_NAME                0 (t)

2 LOAD_CONST               0 (1)

4 DUP_TOP_TWO

6 BINARY_SUBSCR

8 LOAD_CONST               0 (1)

10 BUILD_LIST               1

12 INPLACE_ADD

14 ROT_THREE

16 STORE_SUBSCR

18 LOAD_CONST               1 (None)

20 RETURN_VALUE

Conclusion

Every language has its quirks. By building a deeper understanding of our tools, we improve our chances of avoiding nasty, difficult-to-debug errors. Python is a powerful language that prioritizes elegance and readability. It expects a lot of the developer, and it does not hold your hand as you write. Armed with the knowledge of how our tools work, we can spend more time creating new value and less time pulling out our hair. At least, that's the philosophy at EverQuote.

If you're interested in learning more fun and interesting things about our tech stack, check out our blog. If you're looking to solve interesting problems with a fun and motivated team, EverQuote is hiring: https://careers.everquote.com/ g [add link to careers page]

And remember to always use == and be mindful of closures! :)