# Lab 11: Iterators and Generators

*Due at 11:59pm on Friday, 11/10/2017.*

## Starter Files

Download lab11.zip. Inside the archive, you will find starter files for the questions in this lab, along with a copy of the Ok autograder.

## Submission

By the end of this lab, you should have submitted the lab with
`python3 ok --submit`

. You may submit more than once before the
deadline; only the final submission will be graded.
Check that you have successfully submitted your code on
okpy.org.

- To receive credit for this lab, you must complete Questions 1 - 5 in lab11.py and submit through OK.
- Questions 6 - 10 are
*extra practice*. They can be found in the lab11_extra.py file. It is recommended that you complete these problems on your own time.

## Iterables and Iterators

In lecture, we studied several Python object interfaces, or *protocols*. In this
lab, we will study a new protocol, the *iterator protocol*. Implementing this
protocol allows us to use our objects in for loops! Remember the `for`

loop?
(We really hope so!)

```
for elem in something_iterable:
# do something
```

`for`

loops work on any object that is *iterable*. We previously described it as
working with any sequence -- all sequences are iterable, but there are other
objects that are also iterable! As it turns out, for loops are actually
translated by the interpreter into the following code:

```
the_iterator = iter(something_iterable)
try:
while True:
elem = next(the_iterator)
# do something
except StopIteration:
pass
```

That is, it first calls the built-in `iter`

function to create an *iterator*,
saving it in some new, hidden variable (we've called it `the_iterator`

here).
It then repeatedly calls the built-in `next`

function on this iterator to get
values of `elem`

and stops when that function raises `StopIteration`

.

### Q1: WWPD: Iterators

What would Python display? Try to figure it out before you type it into the interpreter!

Use Ok to test your knowledge with the following "What Would Python Display?" questions:

`python3 ok -q iterators -u`

```
>>> s = [1, [2, [3, [4]]]]
>>> t = iter(s)
>>> next(t)
______1
>>> next(iter(next(t)))
______2
>>> list(t)
______[]
>>> next(map(lambda x: x * x, filter(lambda y: y > 4, range(10))))
______25
>>> r = iter(reversed(range(10000)))
>>> next(r) - next(r)
______1
>>> xs = [2, 3, 4, 5]
>>> y, z = iter(xs), iter(xs)
>>> next(zip(y, z))
______(2, 2)
>>> next(zip(y, y))
______(3, 4)
```

## Generators

A generator function returns a special type of **iterator** called a
**generator** object. Generator functions have `yield`

statements within the
body of the function. Calling a generator function makes it return a generator
object rather than executing the body of the function.

The reason we say a generator object is a special type of **iterator** is that
it has all the properties of an iterator, meaning that:

- Calling the
`iter`

function makes a generator object return itself without modifying its current state. - Calling the
`next`

function makes a generator object compute and return the next object in its sequence. If the sequence is exhausted,`StopIteration`

is raised. - Typically, a generator should not restart unless it's defined that way. But
calling the generator function returns a brand new generator object (like
calling
`iter`

on an iterable object).

However, they do have some fundamental differences:

- An iterator is a class with
`next`

and`iter`

explicitly defined, but a generator can be written as a mere function with a`yield`

in it. `next`

in an iterator uses`return`

, but a generator uses`yield`

.A generator "remembers" its state for the next

`next`

call. Therefore,the first

`next`

call works like this:- Enter the function, run until the line with
`yield`

. - Return the value in the
`yield`

statement, but remember the state of the function for future`next`

calls.

- Enter the function, run until the line with
And subsequent

`next`

calls work like this:- Re-enter the function, start at
**the line after**, and run until the next`yield`

`yield`

statement. - Return the value in the
`yield`

statement, but remember the state of the function for future`next`

calls.

- Re-enter the function, start at

When a generator runs to the end of the function, it raises `StopIteration`

.

Another useful tool for generators is the `yield from`

statement (introduced in
Python 3.3). `yield from`

will yield all values from an iterator or iterable.

### Q2: WWPD: Generators

Use Ok to test your knowledge with the following What would Python Display questions:

`python3 ok -q generators -u`

```
def generator():
print("Starting here")
i = 0
while i < 6:
print("Before yield")
yield i
print("After yield")
i += 1
```

```
>>> g = generator()
>>> g # what type of object is this?
>>> g == iter(g) # equivalent of g.__iter__()
______True
>>> next(g) # equivalent of g.__next__()
______Starting here
Before yield
0
>>> next(g)
______After yield
Before yield
1
>>> next(g)
______After yield
Before yield
2
```

### Q3: Scale

Implement the generator function `scale(s, k)`

, which yields elements of the
given iterable `s`

, scaled by `k`

.

```
def scale(s, k):
"""Yield elements of the iterable s scaled by a number k.
>>> s = scale([1, 5, 2], 5)
>>> type(s)
<class 'generator'>
>>> list(s)
[5, 25, 10]
>>> m = scale(naturals(), 2)
>>> [next(m) for _ in range(5)]
[2, 4, 6, 8, 10]
"""
"*** YOUR CODE HERE ***"
for elem in s:
yield elem * k
```

Use Ok to test your code:

`python3 ok -q scale`

### Q4: Countdown

Write a generator function that counts down to 0.

```
def countdown(n):
"""
A generator that counts down from N to 0.
>>> for number in countdown(5):
... print(number)
...
5
4
3
2
1
0
>>> for number in countdown(2):
... print(number)
...
2
1
0
"""
"*** YOUR CODE HERE ***"
while n >= 0:
yield n
n = n - 1
```

Use Ok to test your code:

`python3 ok -q countdown`

### Q5: Trap

Write a generator function that yields the first `k`

values in iterable `s`

, but
raises a `ValueError`

exception if any more values are requested. You may assume that
`s`

has at least `k`

values.

```
def trap(s, k):
"""Return a generator that yields the first K values in iterable S,
but raises a ValueError exception if any more values are requested.
>>> t = trap([3, 2, 1], 2)
>>> next(t)
3
>>> next(t)
2
>>> next(t)
ValueError
>>> list(trap(range(5), 5))
ValueError
>>> t2 = trap(map(abs, reversed(range(-6, -4))), 2)
>>> next(t2)
5
>>> next(t2)
6
>>> next(t2)
ValueError
"""
"*** YOUR CODE HERE ***"
t = iter(s)
while k > 0:
yield next(t)
k -= 1
raise ValueError("It's a trap!")
```

Use Ok to test your code:

`python3 ok -q trap`

## Extra Questions

### Q6: Hailstone

Write a generator that outputs the hailstone sequence from homework 1.

Here's a quick reminder of how the hailstone sequence is defined:

- Pick a positive integer
`n`

as the start. - If
`n`

is even, divide it by 2. - If
`n`

is odd, multiply it by 3 and add 1. - Continue this process until
`n`

is 1.

```
def hailstone(n):
"""
>>> for num in hailstone(10):
... print(num)
...
10
5
16
8
4
2
1
"""
"*** YOUR CODE HERE ***"
i = n
while i > 1:
yield i
if i % 2 == 0:
i //= 2
else:
i = i * 3 + 1
yield i
```

Use Ok to test your code:

`python3 ok -q hailstone`

### Q7: Repeated

Implement a function (not a generator function) that returns the first value in
iterable `t`

that appears `k`

times in a row.

```
def repeated(t, k):
"""Return the first value in iterable T that appears K times in a row.
>>> s = [3, 2, 1, 2, 1, 4, 4, 5, 5, 5]
>>> repeated(trap(s, 7), 2)
4
>>> repeated(trap(s, 10), 3)
5
>>> print(repeated([4, None, None, None], 3))
None
"""
assert k > 1
"*** YOUR CODE HERE ***"
count = 0
last_item = None
for item in t:
if item == last_item:
count += 1
else:
last_item = item
count = 1
if count == k:
return item
```

Use Ok to test your code:

`python3 ok -q repeated`

### Q8: Merge

Implement `merge(s0, s1)`

, which takes two iterables `s0`

and `s1`

whose
elements are ordered. `merge`

yields elements from `s0`

and `s1`

in sorted
order, eliminating repetition. You may assume `s0`

and `s1`

themselves do not
contain repeats, and that none of the elements of either are `None`

.
You may **not** assume that the iterables are finite; either may produce an infinite
stream of results.

You will probably find it helpful to use the two-argument version of the built-in
`next`

function: `next(s, v)`

is the same as `next(s)`

, except that instead of
raising `StopIteration`

when `s`

runs out of elements, it returns `v`

.

See the doctest for examples of behavior.

```
def merge(s0, s1):
"""Yield the elements of strictly increasing iterables s0 and s1, removing
repeats. Assume that s0 and s1 have no repeats. You can also assume that s0
and s1 represent infinite sequences.
>>> m = merge([0, 2, 4, 6, 8, 10, 12, 14], [0, 3, 6, 9, 12, 15])
>>> type(m)
<class 'generator'>
>>> list(m)
[0, 2, 3, 4, 6, 8, 9, 10, 12, 14, 15]
>>> def big(n):
... k = 0
... while True: yield k; k += n
>>> m = merge(big(2), big(3))
>>> [next(m) for _ in range(11)]
[0, 2, 3, 4, 6, 8, 9, 10, 12, 14, 15]
"""
i0, i1 = iter(s0), iter(s1)
e0, e1 = next(i0, None), next(i1, None)
"*** YOUR CODE HERE ***"
while True:
if e0 is None and e1 is None:
return
elif e0 is None:
yield e1
e1 = next(i1, None)
elif e1 is None:
yield e0
e0 = next(i0, None)
else:
yield min(e0, e1)
if e0 < e1:
e0 = next(i0, None)
elif e1 < e0:
e1 = next(i1, None)
else:
e0, e1 = next(i0, None), next(i1, None)
```

Use Ok to test your code:

`python3 ok -q merge`

### Q9: Remainder Generator

Like functions, generators can also be*higher-order*. For this problem, we will be writing

`remainders_generator`

, which yields a
series of generator objects.
`remainders_generator`

takes in an integer `m`

, and yields `m`

different
generators. The first generator is a generator of multiples of `m`

, i.e. numbers
where the remainder is 0. The second, a generator of natural numbers with
remainder 1 when divided by `m`

. The last generator yield natural numbers with
remainder `m - 1`

when divided by `m`

.

```
def remainders_generator(m):
"""
Takes in an integer m, and yields m different remainder groups
of m.
>>> remainders_mod_four = remainders_generator(4)
>>> for rem_group in remainders_mod_four:
... for _ in range(3):
... print(next(rem_group))
0
4
8
1
5
9
2
6
10
3
7
11
"""
"*** YOUR CODE HERE ***"
def remainder_group(rem):
start = rem
while True:
yield start
start += m
for rem in range(m):
yield remainder_group(rem)
```

Note that if you have implemented this correctly, each of the
generators yielded by `remainder_generator`

will be *infinite* - you
can keep calling `next`

on them forever without running into a
`StopIteration`

exception.

**Hint**: Consider defining an inner generator function. What arguments
should it take in? Where should you call it?

Use Ok to test your code:

`python3 ok -q remainders_generator`

### Q10: Zip Generator

For this problem, we will be writing `zip_generator`

, which yields a
series of lists, each containing the nth items of each iterable.
It should stop when the smallest iterable runs out of elements.

```
def zip_generator(*iterables):
"""
Takes in any number of iterables and zips them together.
Returns a generator that outputs a series of lists, each
containing the nth items of each iterable.
>>> z = zip_generator([1, 2, 3], [4, 5, 6], [7, 8])
>>> for i in z:
... print(i)
...
[1, 4, 7]
[2, 5, 8]
"""
"*** YOUR CODE HERE ***"
iterators = [iter(iterable) for iterable in iterables]
while True:
yield [next(iterator) for iterator in iterators]
```

Use Ok to test your code:

`python3 ok -q zip_generator`