Python Iterators Tutorial

What are iterators?

An iterator in Python is an object that can be traversed. It can be iterated upon and will return data, one element at a time. An object is considered iterable if an iterator is returned from it.

The iterator object implements two special methods, __iter__() and __next__() , which are called the iterator protocol. Most built-in collections like lists, strings, tuples, dictionaries and sets are all iterable objects.

How to iterate through an iterator

To iterate manually through all the items of an iterator we use the next() function, which implements the __next__() special method. When we reach the final iteration and there is no more data to be returned, it will raise a StopIteration error.

shopping_list = ["Milk", "Bread", "Fruits"]

# create an iterator object
list_iter = iter(shopping_list)

# iterate through each next item
print(next(list_iter))
print(next(list_iter))
print(next(list_iter))

In the example above, we create an iterator object with the __iter__() function. Then we iterate manually through each next item one by one with the __next__() function.

If we try to print more than the amount of items in the list, the interpreter will raise an error:

shopping_list = ["Milk", "Bread", "Fruits"]

# create an iterator object
list_iter = iter(shopping_list)

# iterate through each next item
print(next(list_iter))
print(next(list_iter))
print(next(list_iter))

# there is no fourth element in the list
print(next(list_iter))

In the example above, we try to print an element that doesn’t exist. As a result, the interpreter raises a StopIteration error.

    print(next(list_iter))
StopIteration

The interpreter tried to get the next item in the list but couldn’t, because one doesn’t exist so it raises the StopIteration.

How to use a loop to iterate

As we know from the tutorial lessons on any of the collections, a more elegant and efficient way to iterate through objects, is with the for loop.

shopping_list = ["Milk", "Bread", "Fruits"]

for item in shopping_list:
    print(item)

Behind the scenes, the for loop actually creates an iterator and executes the next() function each time the loop runs. The for loop in the example above will be implemented as the following.

shopping_list = ["Milk", "Bread", "Fruits"]

iter_obj = iter(shopping_list)

while True:
    try:
        # try to get the next item
        element = next(iter_obj)
        # for loop body code here
        print(iter_obj)
    # when the last item is reached
    except StopIteration:
        # break out of the loop
        break

The example above seems a bit complex because of the try-except block so let’s go through it step by step:

1. An iterable object is created of the list.
2. In an infinite loop we try to get the next element in the list and do something with it (in this case printing it).
3. When the last item is reached and the StopIteration is raised, we break from the loop.

How to create our own custom iterators (overloading)

To create an object as an iterator we have to implement and override the __iter__() and __next__() functions. We already learned about overloading in the polymorphism tutorial lesson so we’ll only do implementation here and not the concept.

When overloading the __iter__() and __next__() methods we must remember a few details:

__iter__() can do operations, but must always return the iterator object itself.

# define an iteration as a single
# int unit ( 1 )
def __iter__(self):
    self.a = 1
    return self

In the example above our iterator is overloaded to be an int 1.

__next__() can also do operations, but must return a next item.

# skip a number on each iteration
def __next__(self):
    x = self.a
    self.a += 2
    return x

In the example above the next item will be an int of 2 added to the iteration. Combined with the iterator object above, it will return an odd number each time we use __next__() .

class Numbers:
    # define an iteration as a single
    # int unit ( 1 )
    def __iter__(self):
        self.a = 1
        return self

    # skip a number on each iteration
    def __next__(self):
        x = self.a
        self.a += 2
        return x

# create the iterator
odds_iter = iter(Numbers())

# iterate with next()
# is potentially infinite
print(next(odds_iter))
print(next(odds_iter))
print(next(odds_iter))
print(next(odds_iter))
print(next(odds_iter))
print(next(odds_iter))

StopIteration error

If the odd number iterator in the example above were used in a loop, it would turn into an infinite loop. We need some safety measure to break out of the loop.

We can use it almost exactly as a for loop does behind the scenes. Instead of a try except block we will use an if else statement.

class Numbers:
    # define an iteration as a single
    # int unit ( 1 )
    def __iter__(self):
        self.a = 1
        return self

    # skip a number on each iteration
    def __next__(self):
        if self.a <= 10:
            x = self.a
            self.a += 2
            return x
        else:
            raise StopIteration

# create the iterator
odds_iter = iter(Numbers())

# iterate with next()
# is potentially infinite
for i in odds_iter:
    print(i)

In the example above, we added an if statement that only allows the numbers to go up to 10. Anything above that will raise a StopIteration error.

Summary: Points to remember

• Any object that returns an iterator, is considered itarable. Most built-in collections like strings and dictionaries are itarable.
• The iterator object implements the iterator protocol, __iter__() and __next__() .
• Loops create iterators when looping, calling the __next__() function.
• Raise a StopIteration error to break out of a potential infinite loop.
• Custom iterators are created by overloading the __iter__() and __next__() functions.