25

How do you make a repeating generator, like xrange, in Python? For instance, if I do:

>>> m = xrange(5)
>>> print list(m)
>>> print list(m)

I get the same result both times — the numbers 0..4. However, if I try the same with yield:

>>> def myxrange(n):
...   i = 0
...   while i < n:
...     yield i
...     i += 1
>>> m = myxrange(5)
>>> print list(m)
>>> print list(m)

The second time I try to iterate over m, I get nothing back — an empty list.

Is there a simple way to create a repeating generator like xrange with yield, or generator comprehensions? I found a workaround on a Python tracker issue, which uses a decorator to transform a generator into an iterator. This restarts every time you start using it, even if you didn't use all the values last time through, just like xrange. I also came up with my own decorator, based on the same idea, which actually returns a generator, but one which can restart after throwing a StopIteration exception:

@decorator.decorator
def eternal(genfunc, *args, **kwargs):
  class _iterable:
    iter = None
    def __iter__(self): return self
    def next(self, *nargs, **nkwargs):
      self.iter = self.iter or genfunc(*args, **kwargs):
      try:
        return self.iter.next(*nargs, **nkwargs)
      except StopIteration:
        self.iter = None
        raise
  return _iterable()

Is there a better way to solve the problem, using only yield and/or generator comprehensions? Or something built into Python? So I don't need to roll my own classes and decorators?

Update

The comment by u0b34a0f6ae nailed the source of my misunderstanding:

xrange(5) does not return an iterator, it creates an xrange object. xrange objects can be iterated, just like dictionaries, more than once.

My "eternal" function was barking up the wrong tree entirely, by acting like an iterator/generator (__iter__ returns self) rather than like a collection/xrange (__iter__ returns a new iterator).

Community
  • 1
  • 1
Alice Purcell
  • 12,622
  • 6
  • 51
  • 57
  • 2
    Small nitpick, but `xrange()` isn't a generator. `type(xrange(4))` != `type(myxrange(4))`. – John Millikin Sep 03 '09 at 23:44
  • 2
    I think that's more than a small nitpick. That's the entire reason for the difference. And as John pointed out, the desired behavior can be gained with an overloaded __iter__. – ricree Sep 04 '09 at 08:14
  • I'm having a lot of trouble following your implementation code, compared to the other two proposed implementations (one in the Python tracker issue you linked to, the other in @JohnMillikin's answer). In particular, having trouble figuring out: (1) what exactly "@decorator.decorator" means. Can you give a link to doc for it? (2) an example of usage would be very helpful ; in particular, one that exercises args and nargs (3) and can you give an example of how your StopIteration handling adds value? i.e. an example in which your implementation succeeds but the other two implementations fail. – Don Hatch Oct 10 '15 at 08:55
  • @DonHatch Hopefully the update I just added to my question explains why my implementation code is hard to understand (and plain wrong). – Alice Purcell Dec 30 '15 at 21:32

6 Answers6

21

Not directly. Part of the flexibility that allows generators to be used for implementing co-routines, resource management, etc, is that they are always one-shot. Once run, a generator cannot be re-run. You would have to create a new generator object.

However, you can create your own class which overrides __iter__(). It will act like a reusable generator:

def multigen(gen_func):
    class _multigen(object):
        def __init__(self, *args, **kwargs):
            self.__args = args
            self.__kwargs = kwargs
        def __iter__(self):
            return gen_func(*self.__args, **self.__kwargs)
    return _multigen

@multigen
def myxrange(n):
   i = 0
   while i < n:
     yield i
     i += 1
m = myxrange(5)
print list(m)
print list(m)
Gus
  • 4,375
  • 5
  • 31
  • 50
John Millikin
  • 197,344
  • 39
  • 212
  • 226
  • That's basically identical to the workaround I linked to in my question -- I'm guessing you missed it. But thanks! – Alice Purcell Sep 04 '09 at 09:01
  • 2
    "Workaround". There isn't so much to it. xrange(5) does not return an iterator, it creates an xrange object. xrange objects can be iterated, just like dictionaries, more than once. – u0b34a0f6ae Sep 05 '09 at 01:38
14

Using itertools its super easy.

import itertools

alist = [1,2,3]
repeatingGenerator = itertools.cycle(alist)

print(next(generatorInstance)) #=> yields 1
print(next(generatorInstance)) #=> yields 2
print(next(generatorInstance)) #=> yields 3
print(next(generatorInstance)) #=> yields 1 again!
Matt S
  • 1,434
  • 1
  • 15
  • 16
2

If you write a lot of these, John Millikin's answer is the cleanest it gets.

But if you don't mind adding 3 lines and some indentation, you can do it without a custom decorator. This composes 2 tricks:

  1. [Generally useful:] You can easily make a class iterable without implementing .next() - just use a generator for __iter__(self)!

  2. Instead of bothering with a constructor, you can define a one-off class inside a function.

=>

def myxrange(n):
    class Iterable(object):
        def __iter__(self):
            i = 0
            while i < n:
                yield i
                i += 1
    return Iterable()

Small print: I didn't test performance, spawning classes like this might be wasteful. But awesome ;-)

Beni Cherniavsky-Paskin
  • 9,483
  • 2
  • 50
  • 58
0

I think the answer to that is "No". I'm possibly wrong. It may be that with some of the funky new things you can do with generators in 2.6 involving arguments and exception handling that would allow something like what you want. But those features are mostly intended for implementing semi-continuations.

Why do you want to not have your own classes or decorators? And why did you want to create a decorator that returned a generator instead of a class instance?

Omnifarious
  • 54,333
  • 19
  • 131
  • 194
  • (1) Because it takes more code, and seems like something that might have been implemented in some way I didn't know about. (2) Because I initially misunderstood xrange, and thought I wanted an eternal generator rather than an iterable. – Alice Purcell Sep 04 '09 at 09:03
0

You can reset iterators with more_itertools.seekable, a third-party tool.

Install via > pip install more_itertools.

import more_itertools as mit


def myxrange(n):
    """Yield integers."""
    i = 0
    while i < n:
        yield i
        i += 1

m = mit.seekable(myxrange(5))
print(list(m))
m.seek(0)                                              # reset iterator
print(list(m))
# [0, 1, 2, 3, 4]
# [0, 1, 2, 3, 4]

Note: memory consumption grows while advancing an iterator, so be wary wrapping large iterables.

pylang
  • 40,867
  • 14
  • 129
  • 121
-1

use this solution:

>>> myxrange_ = lambda x: myxrange(x)
>>> print list(myxrange_(5))
... [0, 1, 2, 3, 4]
>>> print list(myxrange_(5))
... [0, 1, 2, 3, 4]

>>> for number in myxrange_(5):
...     print number
... 
    0
    1
    2
    3
    4
>>>

and with a decorator:

>>> def decorator(generator):
...     return lambda x: generator(x)
...
>>> @decorator
>>> def myxrange(n):
...   i = 0
...   while i < n:
...     yield i
...     i += 1
...
>>> print list(myxrange(5))
... [0, 1, 2, 3, 4]
>>> print list(myxrange(5))
... [0, 1, 2, 3, 4]
>>>

Simple.

SmartElectron
  • 1,331
  • 1
  • 16
  • 17
  • ¿-1?, don`t understand this. this solution accomplish the requirements of the question. – SmartElectron Nov 14 '13 at 06:16
  • No it does not, because the parameters necessary for repeating the iteration are not encapsulated in the object - they have to be passed every time a new reuse is desired. The entire point of a generator object is to encapsulate the information needed to perform the iteration. A resuable generator should also do this. Note how John Millikin's solution does not require you to type "5" every time you want to use the generator. You type "5" once and the object remembers it for you from that point onwards. – Paul Feb 05 '16 at 18:03
  • In addition, the syntax is not the same as for a generator. Extra parentheses are needed. – Paul Feb 06 '16 at 12:13