How to handle an exhausted iterator?

Question

While searching the Python Documentation I found the equivalent python implementation of Pythons build-in zip() function.

Instead of catching a StopIteration exception which signals that there are no further items produced by the iterator the author(s) use an if statement to check if the returned default value form next() equals object() ("sentinel") and stop the generator:

def zip(*iterables):
    # zip('ABCD', 'xy') --> Ax By
    sentinel = object()
    iterators = [iter(it) for it in iterables]
    while iterators:
        result = []
        for it in iterators:
            elem = next(it, sentinel)

            if elem is sentinel:
                return

            result.append(elem)
        yield tuple(result)

I wonder now if there is any difference between the exception catching or an if statement as used by the Python Docs?

Or better, as @hiro protagonist pointed out:
What's wrong with using a try statement considering EAFP (Easier to ask for forgiveness than permission) in Python?

def zip(*iterables):
    # zip('ABCD', 'xy') --> Ax By
    iterators = [iter(it) for it in iterables]
    while iterators:
        result = []
        for it in iterators:

            try:
                elem = next(it)
            except StopIteration:
                return

            result.append(elem)
        yield tuple(result)

Also as Stoyan Dekov mentioned "A try/except block is extremely efficient if no exceptions are raised. Actually catching an exception is expensive." (see the docs for more information)
But an exception would only occur once, namely as soon as the iterator is exhausted. So exception handling would be the better solution in this case?

`try: elem = next(it); except StopIteration: raise`: catching the exception doesn't add anything useful here, you can just leave it out. — , Jul 30 '15 at 09:43
@Evert: Thanks :) yes thats true. I just copied the snipped form *hiro protagonist* and did what *Dunes* mentioned. Should I except the exception and return or not as mentioned in *PEP 479* ? — elegent, Jul 30 '15 at 09:46
Be sure to note that the code you link to is *equivalent to* not actual. The actual code for `zip` is certainly written in C — dawg, Jul 30 '15 at 10:11
@dawg: Therefor I wrote *"python implementation"*. But yes to make it clear I will add it. — elegent, Jul 30 '15 at 10:27
PEP 479 was new to me as well (kudos to @hiroprotagonist), and from what I read there, yes, catch the exception and `return`. Note that the PEP specifies a transition from Python 3.5 to 3.7 (using a `__future__` import), so it may not make sense with your current Python version. — , Jul 30 '15 at 11:15
The docs aim to be simple to understand, and `if` is simpler that `try/except`. As far as your own code -- use whichever you prefer. ;) — Ethan Furman, Oct 23 '15 at 01:11
@EthanFurman: Thats a good point. Thanks :) But on the other hand the `if`-statement-version is flag-based, (in my opinion) not very pythonic and harder to read, because you don´t really know why the function exits. So why don´t use exception handling when it is present, clearer to read and *explicit*? — elegent, Oct 23 '15 at 08:38
@elegent: As I eluded to in my answer, the docs are read by people of all skill levels, and `if` is learned before `try/except`; so the docs are written as simply as possible. As far as actual coding goes, different people think in different ways, so use whichever method makes more sense to you. The times when LBYL and EAFP actually make a difference is when a race condition can exist -- and there is no such condition here. — Ethan Furman, Oct 23 '15 at 16:07

score 9 · Answer 1 · edited Dec 06 '16 at 11:44

9

You mean as opposed to this?

def zip2(*iterables):
    # zip('ABCD', 'xy') --> Ax By
    iterators = [iter(it) for it in iterables]
    while iterators:
        result = []
        for it in iterators:
            try:
                elem = next(it)
            except StopIteration:
                return

            result.append(elem)
        yield tuple(result)

interesting question... i'd have preferred this alternative version - especially considering EAFP (Easier to ask for forgiveness than permission.)

even if try/except is slower than the if statement; this happens once only - as soon as the first iterator is exhausted.

it may be worth noting that this is not the actual implementaion in python; just an implementation that is equivalent to the real implementation.

UPDATE according to comments:

note that PEP 479 suggests to return from the generator and not raise StopIteration.

edited Dec 06 '16 at 11:44

rrao

297
4
11

answered Jul 30 '15 at 09:06

hiro protagonist

44,693
14
86
111

Don't catch the `StopIteration`. You're in an generator already, let it rise up out of the generator to signify the generator is exhausted. – Dunes Jul 30 '15 at 09:27
3

[PEP 479](https://www.python.org/dev/peps/pep-0479/) suggests otherwise... if i understand correctly not catching it would work for now but cease to work in future versions. – hiro protagonist Jul 30 '15 at 09:30
I'd forgotten about that PEP. However, it's not going to be enforced until version 3.7. – Dunes Jul 30 '15 at 09:44
@hiroprotagonist: I agree. Why do you not change your answer to "a real answer" (Add PEP, delete first line ...)? ;) – elegent Jul 30 '15 at 09:58
@hiroprotagonist: I say thanks for your very informative answer! Keep your good work up! :) – elegent Jul 30 '15 at 14:50

SDekov · Answer 2 · 2015-07-30T09:17:49.833

6

Generally raising exceptions is always considered an expensive operation in any programming language. There are plenty of websites to read why is that and I'm not going to go into details on what it involves.

From Python Docs.

A try/except block is extremely efficient if no exceptions are raised. Actually catching an exception is expensive.

Both using if/else and try/catch has its advantages and disadvantages depending on the situation.

For example try/catch is used mostly for cases where an exception is a rare event (e.g. the code will succeed in almost all cases).
In your example you know the loop will throw an exception every time it is invoked, which makes it very inefficient to use try/catch

edited Jul 30 '15 at 09:17

answered Jul 30 '15 at 09:07

SDekov

9,276
1
20
50

Many thanks for your answer :) But what do you think of *@hiro protagonist* arguments? – elegent Jul 30 '15 at 09:37
His argument is absolutely valid. I can only guess if the author used `if` instead of `StopIteration` exception by accident or he actually considered that the `zip` function will in almost all cases do a very small number of iterations where `if` would be more efficient. Generally I'm used to using Exceptions for ... well Exception situations rather than checks and triggers. – SDekov Jul 30 '15 at 09:49
"*... the author used `if` instead of `StopIteration` exception by accident or he actually considered that the `zip` function will in almost all cases do a very small number of iterations ...*" -- This might be the reason. :D – elegent Jul 30 '15 at 09:53

Ethan Furman · Answer 3 · 2015-10-24T17:48:02.333

When you see equivalent python code in the docs the goal of such code is to be easy to understand. if is easy to understand, while try/except is a higher-level construct.

Functionally, there is no difference. Performance-wise, there may be a small, but probably insignificant, difference.

As far as actual coding goes, different people think in different ways, so use whichever method makes more sense to you. The times when LBYL and EAFP actually make a difference is when a race condition can exist -- and there is no such condition here.

How to handle an exhausted iterator?

3 Answers3