# Lab 6: Nonlocal, Mutability, Iterators and Generators lab06.zip

Due by 11:59pm on Tuesday, March 2.

## Starter Files

Download lab06.zip. Inside the archive, you will find starter files for the questions in this lab, along with a copy of the Ok autograder.

# Topics

Consult this section if you need a refresher on the material for this lab. It's okay to skip directly to the questions and refer back here should you get stuck.

## Nonlocal

We say that a variable defined in a frame is local to that frame. A variable is nonlocal to a frame if it is defined in the environment that the frame belongs to but not the frame itself, i.e. in its parent or ancestor frame.

So far, we know that we can access variables in parent frames:

``````def make_adder(x):
"""
Returns a function which takes in one argument y, and returns x + y.

5
"""
return x + y

Here, when we call `make_adder`, we create a function `adder` that is able to look up the name `x` in `make_adder`'s frame and use its value.

However, we haven't been able to modify variables defined in parent frames. Consider the following function:

``````def make_adder_cumulative(x):
"""
Returns a function which takes in one argument y and returns y + x + the sum of all previous y's.
It also updates x to this new total.

>>> add_all(2) # 3 + 2
5
>>> add_all(5) # 3 + 2 + 5
10
"""
total = x + y
x = total

Here, when we call `make_adder_cumulative`, we create a function `adder` that is able to look up the name `x` in `make_adder_cumulative`'s frame and use its value. However, `adder` also attempts to update the variable `x` in its parent frame. Running this function's doctests, we find that it causes the following error:

``UnboundLocalError: local variable 'x' referenced before assignment``

Why does this happen? When we execute an assignment statement, we are either creating a new binding in our current frame or we are updating an old one in the current frame. For example, the line `x = x + y` in `adder`, is creating the local variable `x` inside `adder`'s frame. This assignment statement tells Python to expect a variable called `x` inside `adder`'s frame, so Python will not look in parent frames for this variable. However, notice that we tried to compute `x + y` before the local variable was created! That's why we get the `UnboundLocalError`.

To avoid this problem, we introduce the `nonlocal` keyword. It allows us to update a variable in a parent frame! The only exception is the global frame. You cannot update a variable in the global frame using `nonlocal`.

Consider this example:

``````def make_adder_cumulative(x):
"""
Returns a function which takes in one argument y and returns y + x + the sum of all previous y's.
It also updates x to this new total.

>>> add_all(2) # 3 + 2
5
>>> add_all(5) # 3 + 2 + 5
10
"""
nonlocal x
total = x + y
x = total

The line `nonlocal x` tells Python that `x` will not be local to this frame, so it will look for it in parent frames. Now we can update `x` without running into problems.

## Mutability

We say that an object is mutable if its state can change as code is executed. The process of changing an object's state is called mutation. Examples of mutable objects include lists and dictionaries. Examples of objects that are not mutable include tuples and functions.

We have seen how to use the `==` operator to check if two expressions evaluate to equal values. We now introduce a new comparison operator, `is`, that checks whether two expressions evaluate to the same values.

Wait, what's the difference? For primitive values, there is none:

``````>>> 2 + 2 == 3 + 1
True
>>> 2 + 2 is 3 + 1
True``````

This is because all primitives have the same identity under the hood. However, with non-primitive values, such as lists, each object has its own identity. That means you can construct two objects that may look exactly the same but have different identities.

``````>>> lst1 = [1, 2, 3, 4]
>>> lst2 = [1, 2, 3, 4]
>>> lst1 == lst2
True
>>> lst1 is lst2
False``````

Here, although the lists referred to by `lst1` and `lst2` have equal contents, they are not the same object. In other words, they are the same in terms of equality, but not in terms of identity.

This is important in our discussion of mutability because when we mutate an object, we simply change its state, not its identity.

``````>>> lst1 = [1, 2, 3, 4]
>>> lst2 = lst1
>>> lst1.append(5)
>>> lst2
[1, 2, 3, 4, 5]
>>> lst1 is lst2
True``````

## Iterators

An iterable is any object that can be iterated through, or gone through one element at a time. One construct that we've used to iterate through an iterable is a for loop:

``````for elem in iterable:
# do something``````

`for` loops work on any object that is iterable. We previously described it as working with any sequence -- all sequences are iterable, but there are other objects that are also iterable! We define an iterable as an object on which calling the built-in function `iter` function returns an iterator. An iterator is another type of object that allows us to iterate through an iterable by keeping track of which element is next in the sequence.

To illustrate this, consider the following block of code, which does the exact same thing as a the for statement above:

``````iterator = iter(iterable)
try:
while True:
elem = next(iterator)
# do something
except StopIteration:
pass``````

Here's a breakdown of what's happening:

• First, the built-in `iter` function is called on the iterable to create a corresponding iterator.
• To get the next element in the sequence, the built-in `next` function is called on this iterator.
• When `next` is called but there are no elements left in the iterator, a `StopIteration` error is raised. In the for loop construct, this exception is caught and execution can continue.

Calling `iter` on an iterable multiple times returns a new iterator each time with distinct states (otherwise, you'd never be able to iterate through a iterable more than once). You can also call `iter` on the iterator itself, which will just return the same iterator without changing its state. However, note that you cannot call `next` directly on an iterable.

Let's see the `iter` and `next` functions in action with an iterable we're already familiar with -- a list.

``````>>> lst = [1, 2, 3, 4]
>>> next(lst)             # Calling next on an iterable
TypeError: 'list' object is not an iterator
>>> list_iter = iter(lst) # Creates an iterator for the list
>>> list_iter
<list_iterator object ...>
>>> next(list_iter)       # Calling next on an iterator
1
>>> next(list_iter)       # Calling next on the same iterator
2
>>> next(iter(list_iter)) # Calling iter on an iterator returns itself
3
>>> list_iter2 = iter(lst)
>>> next(list_iter2)      # Second iterator has new state
1
>>> next(list_iter)       # First iterator is unaffected by second iterator
4
>>> next(list_iter)       # No elements left!
StopIteration
>>> lst                   # Original iterable is unaffected
[1, 2, 3, 4]``````

Since you can call `iter` on iterators, this tells us that that they are also iterables! Note that while all iterators are iterables, the converse is not true - that is, not all iterables are iterators. You can use iterators wherever you can use iterables, but note that since iterators keep their state, they're only good to iterate through an iterable once:

``````>>> list_iter = iter([4, 3, 2, 1])
>>> for e in list_iter:
...     print(e)
4
3
2
1
>>> for e in list_iter:
...     print(e)``````

Analogy: An iterable is like a book (one can flip through the pages) and an iterator for a book would be a bookmark (saves the position and can locate the next page). Calling `iter` on a book gives you a new bookmark independent of other bookmarks, but calling `iter` on a bookmark gives you the bookmark itself, without changing its position at all. Calling `next` on the bookmark moves it to the next page, but does not change the pages in the book. Calling `next` on the book wouldn't make sense semantically. We can also have multiple bookmarks, all independent of each other.

### Iterable Uses

We know that lists are one type of built-in iterable objects. You may have also encountered the `range(start, end)` function, which creates an iterable of ascending integers from start (inclusive) to end (exclusive).

``````>>> for x in range(2, 6):
...     print(x)
...
2
3
4
5``````

Ranges are useful for many things, including performing some operations for a particular number of iterations or iterating through the indices of a list.

There are also some built-in functions that take in iterables and return useful results:

• `map(f, iterable)` - Creates an iterable over `f(x)` for `x` in `iterable`. In some cases, computing a list of the values in this iterable will give us the same result as [`func(x)` for `x` in `iterable`]. However, it's important to keep in mind that iterators can potentially have infinite values because they are evaluated lazily, while lists cannot have infinite elements.
• `filter(f, iterable)` - Creates iterator over `x` for each `x` in `iterable` if `f(x)`
• `zip(iterables*)` - Creates an iterable over co-indexed tuples with elements from each of the `iterables`
• `reversed(iterable)` - Creates iterator over all the elements in the input iterable in reverse order
• `list(iterable)` - Creates a list containing all the elements in the input `iterable`
• `tuple(iterable)` - Creates a tuple containing all the elements in the input `iterable`
• `sorted(iterable)` - Creates a sorted list containing all the elements in the input `iterable`

## Generators

We can create our own custom iterators by writing a generator function, which returns a special type of iterator called a generator. Generator functions have `yield` statements within the body of the function instead of `return` statements. Calling a generator function will return a generator object and will not execute the body of the function.

For example, let's consider the following generator function:

``````def countdown(n):
print("Beginning countdown!")
while n >= 0:
yield n
n -= 1
print("Blastoff!")``````

Calling `countdown(k)` will return a generator object that counts down from `k` to 0. Since generators are iterators, we can call `iter` on the resulting object, which will simply return the same object. Note that the body is not executed at this point; nothing is printed and no numbers are output.

``````>>> c = countdown(5)
>>> c
<generator object countdown ...>
>>> c is iter(c)
True``````

So how is the counting done? Again, since generators are iterators, we call `next` on them to get the next element! The first time `next` is called, execution begins at the first line of the function body and continues until the `yield` statement is reached. The result of evaluating the expression in the `yield` statement is returned. The following interactive session continues from the one above.

``````>>> next(c)
Beginning countdown!
5``````

Unlike functions we've seen before in this course, generator functions can remember their state. On any consecutive calls to `next`, execution picks up from the line after the `yield` statement that was previously executed. Like the first call to `next`, execution will continue until the next `yield` statement is reached. Note that because of this, `Beginning countdown!` doesn't get printed again.

``````>>> next(c)
4
>>> next(c)
3``````

The next 3 calls to `next` will continue to yield consecutive descending integers until 0. On the following call, a `StopIteration` error will be raised because there are no more values to yield (i.e. the end of the function body was reached before hitting a `yield` statement).

``````>>> next(c)
2
>>> next(c)
1
>>> next(c)
0
>>> next(c)
Blastoff!
StopIteration``````

Separate calls to `countdown` will create distinct generator objects with their own state. Usually, generators shouldn't restart. If you'd like to reset the sequence, create another generator object by calling the generator function again.

``````>>> c1, c2 = countdown(5), countdown(5)
>>> c1 is c2
False
>>> next(c1)
5
>>> next(c2)
5``````

Here is a summary of the above:

• A generator function has a `yield` statement and returns a generator object.
• Calling the `iter` function on a generator object returns the same object without modifying its current state.
• The body of a generator function is not evaluated until `next` is called on a resulting generator object. Calling the `next` function on a generator object computes and returns the next object in its sequence. If the sequence is exhausted, `StopIteration` is raised.
• A generator "remembers" its state for the next `next` call. Therefore,

• the first `next` call works like this:

1. Enter the function and run until the line with `yield`.
2. Return the value in the `yield` statement, but remember the state of the function for future `next` calls.
• And subsequent `next` calls work like this:

1. Re-enter the function, start at the line after the `yield` statement that was previously executed, and run until the next `yield` statement.
2. Return the value in the `yield` statement, but remember the state of the function for future `next` calls.
• Calling a generator function returns a brand new generator object (like calling `iter` on an iterable object).
• A generator should not restart unless it's defined that way. To start over from the first element in a generator, just call the generator function again to create a new generator.

Another useful tool for generators is the `yield from` statement (introduced in Python 3.3). `yield from` will yield all values from an iterator or iterable.

``````>>> def gen_list(lst):
...     yield from lst
...
>>> g = gen_list([1, 2, 3, 4])
>>> next(g)
1
>>> next(g)
2
>>> next(g)
3
>>> next(g)
4
>>> next(g)
StopIteration``````

# Required Questions

## Nonlocal WWPD

### Q1: WWPD: Nonlocal Quiz

Use Ok to test your knowledge with the following "What Would Python Display?" questions:

``python3 ok -q nonlocal_quiz -u``

Relevant Topics: Nonlocal

``````>>> def ba(by):
...     def yo(da):
...         by += 2
...         return by
...     return yo(2)
...
>>> ba(3)``````
``````>>> def ba(by):
...     def yo(da):
...         nonlocal by
...         by += 2
...         return by
...     return yo(3)
...
>>> ba(3)``````
``````>>> def ba(by):
...     def yo(da):
...         by.append(da)
...         return by
...     return yo(5)
...
>>> ba([1, 2, 3])``````
``````>>> def ba(by):
...     def yo(da):
...         yoda = by + da
...         return yoda
...     return yo(5)
...
>>> ba(5)``````

## Mutability

### Q2: List-Mutation

Test your understanding of list mutation with the following questions. What would Python display? Type it in the interpreter if you're stuck!

``python3 ok -q list-mutation -u``

Note: if nothing would be output by Python, type `Nothing`. If the code would error, type `Error`.

Relevant Topics: Mutability

``````>>> lst = [5, 6, 7, 8]
>>> lst.append(6)
______None
>>> lst
______[5, 6, 7, 8, 6]
>>> lst.insert(0, 9)
>>> lst
______[9, 5, 6, 7, 8, 6]
>>> x = lst.pop(2)
>>> lst
______[9, 5, 7, 8, 6]
>>> lst.remove(x)
>>> lst
______[9, 5, 7, 8]
>>> a, b = lst, lst[:]
>>> a is lst
______True
>>> b == lst
______True
>>> b is lst
______False``````

### Q3: Insert Items

Write a function which takes in a list `lst`, an argument `entry`, and another argument `elem`. This function will check through each item in `lst` to see if it is equal to `entry`. Upon finding an item equivalent to `entry`, the function should modify the list by placing `elem` into `lst` right after the item. At the end of the function, the modified list should be returned.

See the doctests for examples on how this function is utilized. Use list mutation to modify the original list, no new lists should be created or returned.

Be careful in situations where the values passed into `entry` and `elem` are equivalent, so as not to create an infinitely long list while iterating through it. If you find that your code is taking more than a few seconds to run, it is most likely that the function is in a loop of inserting new values.

``````def insert_items(lst, entry, elem):
"""Inserts elem into lst after each occurence of entry and then returns lst.

>>> test_lst = [1, 5, 8, 5, 2, 3]
>>> new_lst = insert_items(test_lst, 5, 7)
>>> new_lst
[1, 5, 7, 8, 5, 7, 2, 3]
>>> large_lst = [1, 4, 8]
>>> large_lst2 = insert_items(large_lst, 4, 4)
>>> large_lst2
[1, 4, 4, 8]
>>> large_lst3 = insert_items(large_lst2, 4, 6)
>>> large_lst3
[1, 4, 6, 4, 6, 8]
>>> large_lst3 is large_lst
True
"""
``````

Use Ok to test your code:

``python3 ok -q insert_items``

## Iterators and Generators

Generators also allow us to represent infinite sequences, such as the sequence of natural numbers (1, 2, ...) shown in the function below!

Relevant Topics: Iterators and Generators

``````def naturals():
"""A generator function that yields the infinite sequence of natural
numbers, starting at 1.

>>> m = naturals()
>>> type(m)
<class 'generator'>
>>> [next(m) for _ in range(10)]
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
"""
i = 1
while True:
yield i
i += 1``````

### Q4: Scale

Write a generator function `scale(it, multiplier)` which yields the elements of the iterable `it`, multiplied by `multiplier`.

As an extra challenge, try writing this function using a `yield from` statement! A `yield from` statement yields the values from an iterator one at a time.

``````def scale(it, multiplier):
"""Yield elements of the iterable it multiplied by a number multiplier.

>>> m = scale([1, 5, 2], 5)
>>> type(m)
<class 'generator'>
>>> list(m)
[5, 25, 10]

>>> m = scale(naturals(), 2)
>>> [next(m) for _ in range(5)]
[2, 4, 6, 8, 10]
"""
``````

Use Ok to test your code:

``python3 ok -q scale``

### Q5: Hailstone

Write a generator function that outputs the hailstone sequence starting at number `n`.

Here's a quick reminder of how the hailstone sequence is defined:

1. Pick a positive integer `n` as the start.
2. If `n` is even, divide it by 2.
3. If `n` is odd, multiply it by 3 and add 1.
4. Continue this process until `n` is 1.

Note: It is highly encouraged (though not required) to try writing a solution using recursion for some extra practice. Since `hailstone` returns a generator, you can `yield from` a call to `hailstone`!

``````def hailstone(n):
"""Yields the elements of the hailstone sequence starting at n.

>>> for num in hailstone(10):
...     print(num)
...
10
5
16
8
4
2
1
"""
``python3 ok -q hailstone``
``python3 ok --submit``