Why is an iterable object not an iterator?

Question

Here is my code:

from collections import deque

class linehistory:
    def __init__(self, lines, histlen=3):
        self.lines = lines
        self.history = deque(maxlen=histlen)

    def __iter__(self):
        for lineno, line in enumerate(self.lines,1):
            self.history.append((lineno, line))
            yield line

    def clear(self):
        self.history.clear()


f = open('somefile.txt')
lines = linehistory(f)
next(lines)

Error:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
    TypeError: 'linehistory' object is not an iterator

I have no idea why the linehistory object is not an iterator since it has already included __iter__ method in the class.

Answer 1

The concept of iteration is well documented in the Python documentation.

In short, "iterable" is the object which provides the items I want to iterate over. Either it already cintains these items, then it is also called the container. This can be a list, a string, a tuple or anything else which consists of zero to many items. But it also can be an object which produces items, for example one of the many classes contained in itertools. It has __iter__() which returns an iterator.

An "iterator" is the object which is used for one iteration. It can be seen as a kind of "cursor". It has next() (in Python 2) or __next__() (in Python 3) which is called repeatedly until it raises a StopIteration exception. As any iterator is iterable as well (being its own iterator), it also has __iter__() which returns itself.

You can get an iterator for any iterable with iter(obj).

In your example, linehistory (which should be written LineHistory) is iterable as it has an .__iter__(). The generator object created with this is an iterator (as every generator object).

Answer 2

Actually,

All these other answers are wrong (except for @glglgl who has an obtuse style of writing). Your generator function __iter__() would work as is if you called it with a for loop like so

for line in lines:
    print(line)

But because you used next(lines) you have to first use iter() to get the iterator (I presume it just calls __iter__() on the object) like so

it = iter(lines)
print(next(it))

as Mr.Beazley points out

Answer 3

I have no idea why the linehistory object is not an iterator since it has already included __iter__ method in the class.

Wrong. See Iterator Types:

The iterator objects themselves are required to support the following two methods, which together form the iterator protocol:

iterator.__iter__()
Return the iterator object itself. This is required to allow both containers and iterators to be used with the for and in statements. This method corresponds to the tp_iter slot of the type structure for Python objects in the Python/C API.

iterator.__next__()
Return the next item from the container. If there are no further items, raise the StopIteration exception. This method corresponds to the tp_iternext slot of the type structure for Python objects in the Python/C API.

However you can iterate over lines, that's because your __iter__ method is a generator function, see Generator Types:

Python’s generators provide a convenient way to implement the iterator protocol. If a container object’s __iter__() method is implemented as a generator, it will automatically return an iterator object (technically, a generator object) supplying the __iter__() and __next__() methods. More information about generators can be found in the documentation for the yield expression.

Answer 4

Iterator objects need an __iter__ method but they also need to have next implemented:

The iterator objects themselves are required to support the following two methods, which together form the iterator protocol:

iterator.__iter__()
Return the iterator object itself.

iterator.next()
Return the next item from the container.

Python 2.7 Source

In Python 3.x these are the function names:

iterator.__iter__()

iterator.__next__()

Python 3.x Source

Answer 5

Your object it is not an iterator the same way a list it is not an iterator, but an iterable. You could make it an iterator, though. Because an iterator is an object in its own.

help(list)

Then:

 |  __iter__(self, /)
 |      Implement iter(self).

Suppose you have a list:

a = [1,2,3]

and you try to call

next(a)

You get:

Traceback (most recent call last):
  File "<pyshell>", line 1, in <module>
TypeError: 'list' object is not an iterator

But you can make an iterator of a list by applying an iter() on it.

>>> iter_a = iter(a)
>>> print(iter_a)
<list_iterator object at 0x03FE8FB0>

>>> next(iter_a)
1

So do this with your code:

f = open('somefile.txt')
lines = linehistory(f)
lines_iter = lines.__iter__()
print(next(lines_iter))

First line of somefile.txt (my file):

>>> %Run 'some file iterator.py'
aaaaaaa

Now do:

>>> dir(lines_iter)

You will see:

 '__iter__', '__le__', '__lt__', '__name__', '__ne__', '__new__', '__next__', 

See? It now has a next method!

Answer 6

The Python "TypeError: 'list' object is not an iterator" occurs when we try to use a list as an iterator. To solve the error, pass the list to the iter() function to get an iterator, e.g. my_iterator = iter(my_list).

Here is an example of how the error occurs.

my_list = ['a', 'b', 'c']

# ⛔️ TypeError: 'list' object is not an iterator
print(next(my_list))

We tried to use the list as an iterator, but lists are not iterators (they are iterable).

To solve the error, pass the list to the iter() function to get an iterator.

my_list = ['a', 'b', 'c']

my_iterator = iter(my_list)

print(next(my_iterator)) # ️ "a"
print(next(my_iterator)) # ️ "b"
print(next(my_iterator)) # ️ "c"

The iter() function returns an iterator.

An iterator object represents a stream of data. Every time we pass the iterator to the next() function, the next item in the stream is returned.

When the iterator is exhausted, the StopIteration exception is raised.

my_list = ['a', 'b', 'c']

my_iterator = iter(my_list)


try:
    print(next(my_iterator))  # ️ "a"
    print(next(my_iterator))  # ️ "b"
    print(next(my_iterator))  # ️ "c"
    print(next(my_iterator))
except StopIteration:
    # ️ this runs
    print('iterator is exhausted')

When you use a list in a for loop an iterator is create for you automatically.

my_list = ['a', 'b', 'c']

for el in my_list:
    print(el)  # ️ a, b, c

Iterators are required to have an iter() method that returns the iterator object itself.

Every iterator is also an iterable, however not every iterable is an iterator.

Examples of iterables include all sequence types (list, str, tuple) and some non-sequence types like dict, file objects and other objects that define an iter() or a getitem() method.

When an iterable is passed as an argument to the iter() function, it returns an iterator. However, it is usually not necessary to use the iter() function because a for statement automatically does that for us.

When you use a for statement or pass an iterable to the iter() function, a new iterator is created each time.

On the other hand, you can only iterate over an iterator once before it is exhausted and appears as an empty container. If you need to check if a value is iterable, use a try/except statement.

The iter() function raises a TypeError if the passed in value doesn't support the iter() method or the sequence protocol (the getitem() method).

If we pass a non-iterable object like an integer to the iter() function, the except block is run.

my_str = 'hello'
# my_str = 1  # raise exception

try:
    my_iterator = iter(my_str)

    for i in my_iterator:
        print(i)  # ️ h, e, l, l, o
except TypeError as te:
    print(te)

The iter() function raises a TypeError if the passed in value doesn't support the iter() method or the sequence protocol (the getitem() method).

If we pass a non-iterable object like an integer to the iter() function, the except block is run.

ref: https://bobbyhadz.com/blog/python-typeerror-list-object-is-not-an-iterator