21

I found that:

>>> a={'x':42, 'y':3.14, 'z':7}
>>> b=a.__iter__()
>>> b.__dir__()
['__next__', ..., '__iter__', ...]
>>> b
<set_iterator object at 0x7efdd4e5afc0>

Does an iterator always have the __iter__ method?

According to https://stackoverflow.com/a/9884259 an iterator also an iterable. If it is true that an iterator always has __iter__ method?

MSeifert
  • 145,886
  • 38
  • 333
  • 352
  • 1
    Yes, it always does. And it *should return `self`*. – juanpa.arrivillaga Sep 07 '17 at 22:52
  • Seriously, though, check out [this gist](https://gist.github.com/juanarrivillaga/e6a68d70cbae3d2ca72f3d81f869d4f6). I think it should clear up some confusion with regards to for loops, what is an iterator, and what is an iterable. – juanpa.arrivillaga Sep 07 '17 at 23:25
  • I think it's worth noting that sometimes people's language about iterators and iterables is the reverse of the actual protocol details. That is, if somebody says "X is an iterable", they may intend to exclude iterators (despite iterators always having an `__iter__` method and so being iterable in the official sense). There's no good single-word term for "non-iterator iterable", so some folks just use "iterable" for that, even though its not accurate. I elaborated a bit on this topic in [a previous answer](https://stackoverflow.com/a/40324491/1405065). – Blckknght Sep 07 '17 at 23:43

4 Answers4

18

An iterable needs to implement an __iter__ method or a __getitem__ method:

An object can be iterated over with for if it implements __iter__() or __getitem__().

An iterator needs a __iter__ method (that returns self) and a __next__ method (I'm not 100% sure about the __next__).

it is true that an iterator always has __iter__ method?

Yes!

This is also documented in the Data model:

object.__iter__(self)

This method is called when an iterator is required for a container. This method should return a new iterator object that can iterate over all the objects in the container. For mappings, it should iterate over the keys of the container.

Iterator objects also need to implement this method; they are required to return themselves. For more information on iterator objects, see Iterator Types.

(Emphasis mine)

As to your second question:

Is an iterator also an iterable?

Yes, because it has a __iter__ method.

Additional notes

Besides the formal implementation it's easy to check if something is iterable by just checking if iter() can be called on it:

def is_iterable(something):
    try:
        iter(something)
    except TypeError:
        return False
    else:
        return True

Likewise it's possible to check if something is an iterator by checking if iter() called on something returns itself:

def is_iterator(something):
    try:
        return iter(something) is something  # it needs to return itself to be an iterator
    except TypeError:
        return False

But don't use them in development code, these are just for "visualization". Mostly you just iterator over something using for ... in ... or if you need an iterator you use iterator = iter(...) and then process the iterator by calling next(iterator) until it throws a StopIteration.

Community
  • 1
  • 1
MSeifert
  • 145,886
  • 38
  • 333
  • 352
  • Thanks. Which reference can I find "An iterable has an __iter__ method or a __getitem__ and __len__ method"? What are those iterables which don't have `__iter__`? –  Sep 07 '17 at 22:56
  • @Ben I added some links :) – MSeifert Sep 07 '17 at 23:00
  • Thanks. What are those builtin iterables which don't have `__iter__`? –  Sep 07 '17 at 23:07
  • I don't think there are any. That's mostly used for user-defined sequence-like iterables. – MSeifert Sep 07 '17 at 23:09
  • Is it possible to convert iterables which don't have `__iter__` into iterators? –  Sep 07 '17 at 23:11
  • @Ben Any specific use-case? Generally you don't need iterators often, you just iterate over whatever you have and then see if it works. – MSeifert Sep 07 '17 at 23:15
  • 3
    @Ben: In Python 2, `str` doesn't have an `__iter__`. I'm not aware of any built-in iterable types with no `__iter__` on Python 3. – user2357112 Sep 07 '17 at 23:22
  • @Ben you can always get an iterator from *any iterable* by using `iter`, i.e. `iterator = iter(my_iterable)` – juanpa.arrivillaga Sep 07 '17 at 23:23
7

An iterator is iterable. And yes, an iterator always has an __iter__ method.

Calling iter on an iterator, which summons the __iter__ hook, returns the same iterator:

>>> it = iter([]) # return iterator from iterable
>>> it is iter(iter(it)) is it.__iter__().__iter__().__iter__()
True

A classical example of method chaining.

As you must have also noticed, most implementations of the iterator protocol for custom classes always follows:

def __iter__(self):
    return self

That is if the iteration is not delegated to another iterator, via say return iter(...).

It would be quite counter-intuitive for an iterator to not implement the iterator protocol don't you think? The __iter__ implementation of the protocol is described below:

iterator.__iter__()

Return the iterator object itself. This is required to allow both containers and iterators to be used with the for and in statements.

[Emphasis mine]

This behaviour is expectedly consistent with iterator objects returned by builtins:

>>> m = map(None, [])
>>> m
<map object at 0x...>
>>> m is m.__iter__().__iter__().__iter__()
True

P.S: I apologise for calling dunders repeatedly; makes it look like it's the right thing to do. But it's not!

Moses Koledoye
  • 77,341
  • 8
  • 133
  • 139
  • No. An iterator could have `__getitem__` instead of `__iter__` so this is not true. – RobertB Sep 07 '17 at 22:54
  • 2
    No, you're thinking about "iterables". "Iterators" always have a `__iter__` method. – MSeifert Sep 07 '17 at 22:55
  • *This is required to allow both containers and iterators to be used with the for and in statements* <-- that seems to be a documentation bug. A container can just implement `__contains__`, and then `__iter__` is *not* required for it to be used on the right hand side of `in` statement. – wim Sep 07 '17 at 23:35
  • @wim I think they mean the for-loop syntax: `for ... in ...` with "`for` and `in`". Not actually the `in` as in `contains`. – MSeifert Sep 07 '17 at 23:53
  • I doubt it. The looping is just called a "for statement" in the grammar. And that text is [from 16 years ago](https://github.com/python/cpython/commit/93656e76f9d7f11af13f5dc0ccf7b2051033fa29) (!), so the intention was probably to describe old-style membership tests via iteration. Otherwise there is no good reason to explicitly write "the `for` and `in` statements" as two distinct statements like that. – wim Sep 08 '17 at 02:31
  • Also, the `in` statement in the section under question is actually linked to [Membership test operations](https://docs.python.org/3/reference/expressions.html#membership-test-operations). – wim Sep 08 '17 at 03:50
1

An iterator is iterable.

That's documented and explained here:

Iterators are required to have an __iter__() method that returns the iterator object itself so every iterator is also iterable

An iterable is not necessarily an iterator

Iterators must have a __next__ method, by definition. To give a simple counterexample:

>>> ''.__next__
AttributeError: 'str' object has no attribute '__next__'

A string object is an iterable, but is not an iterator.

wim
  • 338,267
  • 99
  • 616
  • 750
  • Thanks. "A string object is an iterable, but is not an iterator. " but `'abc'.__iter__` exists and `'abc'` is a string object. Do I miss something? –  Sep 07 '17 at 23:09
  • Iterators return themself when `__iter__` is called. But strings return an `string_iterator` object. So it's iterable but not an iterator (it didn't return itself). – MSeifert Sep 07 '17 at 23:11
  • @MSeifert Is `string_iterator` object not an iterator? –  Sep 07 '17 at 23:12
  • `string_iterator` is an iterator but `str` isn't (`str` is just iterable). – MSeifert Sep 07 '17 at 23:12
  • 2
    @Ben **because to be an iterator it needs a `__next__` method**. – juanpa.arrivillaga Sep 07 '17 at 23:15
  • @MSeifert Thanks . I see. What are those builtin iterables which are not iterators, besides `str`? –  Sep 07 '17 at 23:17
  • All builtin containers are iterable but not iterators: `list`, `tuple`, `dict`, `set`, `frozenset`, `bytearray`, `range` ... But there are some functions that are iterators: `map`, `zip`, `enumerate`, `reversed`. – MSeifert Sep 07 '17 at 23:18
  • @MSeifert mmmm just to nitpick - the *functions* aren't iterators, but they *return iterators*. – juanpa.arrivillaga Sep 07 '17 at 23:25
  • @juanpa.arrivillaga What functions? [`map`](https://github.com/python/cpython/blob/v3.6.2/Python/bltinmodule.c#L1254-L1302), etc. are classes. – MSeifert Sep 07 '17 at 23:26
  • @MSeifert fine, *callables* then. You were the one who said "but there are some functions that are iterators: `map` etc...". My point is, `map` isn't an iterator, it is a *callable* that returns a map-obect, which is an iterator. Similarly, one wouldn't say `open` is an iterator, but it returns an iterator. – juanpa.arrivillaga Sep 07 '17 at 23:28
  • 2
    @juanpa.arrivillaga No, they are really classes. Just try: `map.__iter__` and `map.__next__`. I'm not saying `open` or `iter` are iterators. Those two are truly factory functions that return iterators. – MSeifert Sep 07 '17 at 23:33
  • @MSeifert oh wow, I had never noticed, but now it makes sense. – juanpa.arrivillaga Sep 07 '17 at 23:37
  • @MSeifert I like your reverse nit-pick :) However, I think what juanpa was pointing out is that *functions that are iterators* should have read *Built-in functions/classes that return iterators* e.g. `map.__new__` returns the iterator. – Moses Koledoye Sep 07 '17 at 23:39
  • Well you could say that about any `__new__` method. I think, except for `reversed`, the `__new__` methods of these classes aren't used as factory methods (return not-instances) though. – MSeifert Sep 07 '17 at 23:46
0

Does an iterator always have the __iter__ method?

Yes. All iterators have an __iter__ method that returns itself. From the docs

iterator.__iter__()

Return the iterator object itself. This is required to allow both containers and iterators to be used with the for and in statements.

countunique
  • 4,068
  • 6
  • 26
  • 35