How can I get generators/iterators to evaluate as False when exhausted?

Question

Other empty objects in Python evaluate as False -- how can I get iterators/generators to do so as well?

Answer 1

Guido doesn't want generators and iterators to behave that way.

Objects are true by default. They can be false only if they define __len__ that returns zero or __nonzero__ that returns False (the latter is called __bool__ in Py3.x).

You can add one of those methods to a custom iterator, but it doesn't match Guido's intent. He rejected adding __len__ to iterators where the upcoming length is known. That is how we got __length_hint__ instead.

So, the only way to tell if an iterator is empty is to call next() on it and see if it raises StopIteration.

On ASPN, I believe there are some recipes using this technique for lookahead wrapper. If a value is fetched, it is saved-up the an upcoming next() call.

Answer 2

By default all objects in Python evaluate as True. In order to support False evaluations the object's class must have either a __len__ method (0 -> False), or a __nonzero__ method (False -> False). Note: __nonzero__ ==> __bool__ in Python 3.x.

Because the iterator protocol is intentionally kept simple, and because there are many types of iterators/generators that aren't able to know if there are more values to produce before attempting to produce them, True/False evaluation is not part of the iterator protocol.

If you really want this behavior, you have to provide it yourself. One way is to wrap the generator/iterator in a class that provides the missing functionality.

Note that this code only evaluates to False after StopIteration has been raised.

As a bonus, this code works for pythons 2.4+

try:
    next
except NameError:       # doesn't show up until python 2.6
    def next(iter):
        return iter.next()

Empty = object()

class Boolean_Iterator(object):
    """Adds the abilities
    True/False tests:  True means there /may/ be items still remaining to be used
    """
    def __init__(self, iterator):
        self._iter = iter(iterator)
        self._alive = True
    def __iter__(self):
        return self
    def __next__(self):
        try:
            result = next(self._iter)
        except StopIteration:
            self._alive = False
            raise
        return result
    next = __next__                     # python 2.x
    def __bool__(self):
        return self._alive
    __nonzero__ = __bool__              # python 2.x

If you also want look-ahead (or peek) behavior, this code will do the trick (it evaluates to False before StopIteration is raised):

try:
    next
except NameError:       # doesn't show up until python 2.6
    def next(iter):
        return iter.next()

Empty = object()

class Iterator(object):
    """Adds the abilities
    True/False tests:  True means there are items still remaining to be used
    peek(): get the next item without removing it from the sequence
    """
    def __init__(self, iterator):
        self._iter = iter(iterator)
        self._peek = Empty
        self.peek()
    def __next__(self):
        peek, self._peek = self._peek, Empty
        self.peek()
        if peek is not Empty:
            return peek
        raise StopIteration
    next = __next__                     # python 2.x
    def __bool__(self):
        return self._peek is not Empty
    __nonzero__ = __bool__              # python 2.x
    def peek(self):
        if self._peek is not Empty:
            return self._peek
        self._peek = next(self._iter, Empty)
        return self._peek

Keep in mind that peek behaviour is not appropriate when the timing of the underlying iterator/generator is relevant to its produced values.

Also keep in mind that third-party code, and possibly the stdlib, may rely on iterators/generators always evaluating to True. If you want peek without bool, remove the __nonzero__ and __bool__ methods.

Answer 3

an 'empty thing' is automatically not an iterator. containers can be empty or not, and you can get iterators over the containers, but those iterators are not falsey when exhausted.

A good example of why iterators don't become falsey is sys.stdin. The problem with making sys.stdin falsey when it reaches the end of input is that there's no way of actually knowing if you have reached the end of such a stream without trying to consume input from it. The main reason for wanting an iterator to be falsey would be to 'peek' to see if getting the next item would be valid; but for sys.stdin, that's obviously not practical.

here's another example

(x for x in xrange(1000) if random.randrange(0, 2))

there's no way of knowing if this generator will return any more numbers without doing a bunch of work, you actually have to find out what the next value will be.

The solution is to just get the next value from the iterator. If it is empty, your loop will exit, or you'll get a StopIteration exception if you're not in a loop.