9

First I want to clarify, I'm NOT asking what is "iterator".

This is how the term "iterable" is defined in Python's doc:

iterable

An object capable of returning its members one at a time. Examples of iterables include all sequence types (such as list, str, and tuple) and some non-sequence types like dict, file objects, and objects of any classes you define with an __iter__() or __getitem__() method.

Iterables can be used in a for loop and in many other places where a sequence is needed (zip(), map(), ...). When an iterable object is passed as an argument to the built-in function iter(), it returns an iterator for the object. This iterator is good for one pass over the set of values. When using iterables, it is usually not necessary to call iter() or deal with iterator objects yourself. The for statement does that automatically for you, creating a temporary unnamed variable to hold the iterator for the duration of the loop.

See also iterator, sequence, and generator.

As other people suggested, using isinstance(e, collections.Iterable) is the most pythonic way to check if an object is iterable.
So I did some test with Python 3.4.3:

from collections.abc import Iterable

class MyTrain:
    def __getitem__(self, index):
        if index > 3:
            raise IndexError("that's enough!")

        return index

for name in MyTrain():
    print(name)  # 0, 1, 2, 3

print(isinstance(MyTrain(), Iterable))  # False

The result is quite strange: MyTrain has defined __getitem__ method, but it is not considered as an iterable object, not to mention it's capable of returning one number at a time.

Then I removed __getitem__ and added the __iter__ method:

from collections.abc import Iterable

class MyTrain:    
    def __iter__(self):
        print("__iter__ called")
        pass

print(isinstance(MyTrain(), Iterable))  # True

for name in MyTrain():
    print(name)  # TypeError: iter() returned non-iterator of type 'NoneType'

It is now considered as a "true" iterable object in spite of it cannot produce anything while iterating.

So did I misunderstand something or is the documentation incorrect?

laike9m
  • 18,344
  • 20
  • 107
  • 140
  • 7
    `isinstance` won't check that the interface is implemented correctly, that doesn't get found out until you actually try to iterate over it, just that the appropriate method(s) (in this case [only `__iter__`](https://docs.python.org/2/library/collections.html#collections.Iterable)) are available. – jonrsharpe Sep 26 '15 at 17:33
  • 1
    https://docs.python.org/3/library/collections.abc.html#collections.abc.Iterable – Padraic Cunningham Sep 26 '15 at 17:35
  • 12
    *"using `isinstance(e, collections.Iterable)` is the most pythonic way to check if an object is iterable"* - no, I would say that **trying to iterate over it** is the most Pythonic way! – jonrsharpe Sep 26 '15 at 17:36
  • Jonrsharpe has it. Easier to ask forgiveness than permission. – Nick Bailey Sep 26 '15 at 17:54
  • 1
    to test if something is an iterable, I do a try / except block where I try var = iter(var), if it throws and exception, then it's not an iterable – DevLounge Sep 26 '15 at 17:58
  • 2
    @jonrsharpe. But that might be self-defeating, since it could consume the iterable. Maybe a better check would be to try `iter(x)` and see if it raises a `TypeError`. – ekhumoro Sep 26 '15 at 18:03
  • @ekhumoro but if it's iterable then you *want* to consume it, and if it's not then you *can't*. It's hard to see how that could go too far wrong. – jonrsharpe Sep 26 '15 at 18:10
  • @jonrsharpe Then what's `isinstance(e, collections.Iterable)` all about? – laike9m Sep 26 '15 at 18:21
  • @laike9m what do you mean *"what's [it] all about"*? – jonrsharpe Sep 26 '15 at 18:22
  • @jonrsharpe What will you use it for? – laike9m Sep 26 '15 at 18:25
  • @jonrsharpe. It will go wrong if you need to find out whether an object is iterable, but without there being any side-effects (such as consuming it). – ekhumoro Sep 26 '15 at 18:25
  • @ekhumoro my point is, if you're using duck typing then you would never actually *"find out whether an object is iterable"* **other than by (trying to) iterate over it**. It's not like you'd do `if is_iterable(thing): for whatever in thing:` then implement `is_iterable` also with `for whatever in thing:`! – jonrsharpe Sep 26 '15 at 18:26
  • @laike9m I'm not sure that you *would* use it; as I commented already, the Pythonic way to find out if something is iterable is to *try to iterate over it*. – jonrsharpe Sep 26 '15 at 18:27
  • @jonrsharpe. Sure, but there are other reasons for checking without consuming. For instance, your code may need to perform some setup actions which conditionally depend on whether the object is iterable or not. Using `iter` is really exactly the same as "just try to iterate over it", but with the additional benefit of not consumimg the iterator first. – ekhumoro Sep 26 '15 at 18:45
  • @ekhumoro fair enough; although I've never found myself in that situation, you're right that using `iter` would let you check in those circumstances. – jonrsharpe Sep 26 '15 at 18:47

3 Answers3

5

I think the point of confusion here is that, although implementing __getitem__ does allow you to iterate over an object, it isn't part of the interface defined by Iterable.

The abstract base classes allow a form of virtual subclassing, where classes that implement the specified methods (in the case of Iterable, only __iter__) are considered by isinstance and issubclass to be subclasses of the ABCs even if they don't explicitly inherit from them. It doesn't check whether the method implementation actually works, though, just whether or not it's provided.

For more information, see PEP-3119, which introduced ABCs.

using isinstance(e, collections.Iterable) is the most pythonic way to check if an object is iterable

I disagree; I would use duck-typing and just attempt to iterate over the object. If the object isn't iterable a TypeError will be raised, which you can catch in your function if you want to deal with non-iterable inputs, or allow to percolate up to the caller if not. This completely side-steps how the object has decided to implement iteration, and just finds out whether or not it does at the most appropriate time.

To add a little more, I think the docs you've quoted are slightly misleading. To quote the iter docs, which perhaps clear this up:

object must be a collection object which supports the iteration protocol (the __iter__() method), or it must support the sequence protocol (the __getitem__() method with integer arguments starting at 0).

This makes it clear that, although both protocols make the object iterable, only one is the actual "iteration protocol", and it is this that isinstance(thing, Iterable) tests for. Therefore we could conclude that one way to check for "things you can iterate over" in the most general case would be:

isinstance(thing, (Iterable, Sequence))

although this does also require you to implement __len__ along with __getitem__ to "virtually sub-class" Sequence.

jonrsharpe
  • 115,751
  • 26
  • 228
  • 437
  • Thx jorsharpe, your answer is of great help. For now, I really think the doc I quoted is misleading. The glossary "iterable" in the glossary doc means "can be iterated" no matter how it's done, while Python's `collections.abc.Iterable` represents valid iteration container(I made this word up), that is whatever object that defines the `__iter__` method. – laike9m Sep 27 '15 at 05:24
  • So `collections.abc.Iterable` is not necessarily "iterable" since it could return an invalid iterator which does not implement `__next__`. Meanwhile , an "iterable" object could have no relation with `collections.abc.Iterable` by only defining `__getitem__`. What do you think? – laike9m Sep 27 '15 at 05:32
  • @laike9m that's right, but then simply defining a `__getitem__` method is also no guarantee that it actually *works*! This is why I prefer the duck typing [*"EAFP"*](https://docs.python.org/3/glossary.html#term-eafp) approach - given that you can't truly know if an object is *actually iterable* until you do it, then that's the most appropriate time to deal with it not being. Ideally the documentation should say something like *"an object is iterable if it implements the iterator or sequence protocols"*, perhaps, although it seems that `__len__` isn't needed in practice. – jonrsharpe Sep 27 '15 at 08:49
1

It is an iterable. However you haven't inherited from abc.Iterable, so naturally Python won't report it as being descended from that class. The two things -being an iterable, and descending from that base class - are quite separate.

Daniel Roseman
  • 588,541
  • 66
  • 880
  • 895
  • I think what's confused the OP is that classes implementing `__iter__` are virtual subclasses of `Iterable` even if they don't explicitly inherit from it. – jonrsharpe Sep 26 '15 at 18:12
-1

Iterable is something(collection anything) that allows some kind of iteration on its elements. But what is the generic way of iteration in python? That is using - in keyword, which uses __iter__ method of an object. So, in that terms any object that defines __iter__ can be used with in, and is an Iterable.

So, most 'duck-typish' way to check if an object is iterable is if an object is this, (Yeah, I know implicitly that is what's happening in isinstance case as well, due to virtual classes)

hasattr(train, '__iter__')

because according to duck-typing, we care about behavior provided by an object instead of its ancestory.

If you have a faulty implementation of __iter__ that doesn't means object is not iterable, it just means that you have bug in your code.

Note:- Those objects that don't define __iter__ can still be iterable in general sense, by using some other method, it's just they can't be used with in keyword. E.g.:- NumberList instance is iterable over each method, but isn't iterable in python sense.

class NumberList:

     def __init__(self, values):
         self.values = values

     def each(self):
         return self.values
hspandher
  • 15,934
  • 2
  • 32
  • 45
  • 1. That's **not** duck typing. 2. `hasattr(train, '__iter__')` is roughly what `isinstance(train, Iterable)` already does. – jonrsharpe Sep 26 '15 at 18:16
  • @jonrshare I know, but that is bit misleading. Don't you think isinstance should check inheritance instead – hspandher Sep 26 '15 at 18:17
  • If an object provides '__iter__' method, we don't care whether it's instance of Iterable or not. Isn't that what duck typing means – hspandher Sep 26 '15 at 18:18
  • *"Don't you think isinstance should check inheritance instead"* - no, I think it should do what it's documented to: [*"Return true if the object argument is an instance of the classinfo argument, or of a (direct, indirect or virtual) subclass thereof."*](https://docs.python.org/3/library/functions.html#isinstance) (where for virtual see https://docs.python.org/3/glossary.html#term-abstract-base-class) – jonrsharpe Sep 26 '15 at 18:18
  • [Duck typing](https://docs.python.org/3/glossary.html#term-duck-typing) means not using `hasattr` at all, just writing `for thing in train:` and either dealing with the error in the function or letting the caller catch it. – jonrsharpe Sep 26 '15 at 18:19
  • What you are saying is, if I define '__iter__' on some method, python implicitly consider it inheriting from abstract class, thereby making it instance of Iterable. So, what isinstance is doing is almost the same thing as checking hasattr. Don't you think that goes bit against "Explicit is better than implicit." – hspandher Sep 26 '15 at 18:23
  • If he were using `in`, we wouldn't be having this discussion. All I'm saying is checking `hasattr` looks better than checking isinstance from dynamic language point of view – hspandher Sep 26 '15 at 18:26