iterator.md

Iterator

1. Brief
2. Iterable
   1. Definition
   2. Custom iterable (__iter__)
3. Iterator
   1. iter(), next()
   2. iter() with sentinel
   3. Custom iterator (__iter__, __next__)
4. Generator
   1. yield
   2. Generator expression
5. Iterators are lazy

---

Brief

A process that is repeated more than one time by applying the same logic is called an Iteration.

---

iterable

Definition

An object capable of returning its members one at a time:

1. Sequence types: (have __getitem__() and __len__() methods)
   • list, str, tuple, ...
2. Non-sequence types:
   • dict, file objects, ...
   • object (class) with __iter__() method
   • generators

Custom iterable

Object must have __iter__ method

class Foo(object):
    def __iter__(self):
        return (x for x in range(4))

it = iter(Foo())       # it = <generator object Foo.__iter__.<locals>.<genexpr> at 0x...>
l = list(Foo())        # l = [0, 1, 2, 3]
for i in Foo():
    print(i, end=' ')  # output: 0 1 2 3

---

iterator

iter(), next()

Iterable passed to iter() function returns iterator

i = iter([1, 2, 3])
v = next(i)  # v = 1
v = next(i)  # v = 2
v = next(i)  # v = 3
v = next(i)  # StopIteration exception

One can use for in to loop over iterator

i = iter([1, 2, 3])
for x in i:
    print(x, end=' ')  # output: 1 2 3 

One don't have to use iter() on iterable when using for in to loop over. It does it automatically !!!

i = [1, 2, 3]
for x in i:
    print(x, end=' ')  # output: 1 2 3 

iter() with sentinel

Something like while True with break

from functools import partial
with open('mydata.db', 'rb') as f:
    for block in iter(partial(f.read, 64), b''):
        process_block(block)

Custom iterator: __iter__(), __next__()

Can have __init__ like other casses but must have methods:

1. __iter__ - has to return self + some initialisation if __init__ is omitted
2. __next__ - return next item in the sequence

class Count:
    """By default, counts from 0 to 10"""
    def __init__(self, start=0, stop=10):
        self.num = start
        self.stop = stop

    def __iter__(self):
        return self

    def __next__(self):
        if self.num <= self.stop:
            num = self.num
            self.num += 1
            return num
        else:
            raise StopIteration

Usage:

c = Count()
for i in c:
    print(i, end=' ')  # output: 0 1 2 3 4 5 6 7 8 9 10

---

Generator

yield

Generator - easy way of making iterator. Instead of class iterator use function with yield
Simple generator:

def gen():
    for i in range(10):
        yield i ** 2

for x in gen():
    print(x, end=' ')  # output: 0 1 4 9 16 25 36 49 64 81

Generator expression

Similar to list, dict, set comprehensions but returns generator !!! and takes less memory but takes more time !

import sys
g = (i ** 2 for i in range(10000))  # type(g) = <class 'generator'>
s = sys.getsizeof(g)                # s = 112

import sys
l = [i ** 2 for i in range(10000)]  # type(l) = <class 'list'>
s = sys.getsizeof(l)                # s = 87616

---

iterators are lazy

Iterators providing one value at a time - it saves memory. But not time. Good for large amount of data, reading large files or infinitely long iterables.