155

I need to iterate over a circular list, possibly many times, each time starting with the last visited item.

The use case is a connection pool. A client asks for connection, an iterator checks if pointed-to connection is available and returns it, otherwise loops until it finds one that is available.

How can I do this neatly in Python?


If you instead need an immediately created list of the results up to a certain length, rather than iterating on demand: see Repeat list to max number of elements for general techniques, and How to replicate array to specific length array for Numpy-specific techniques.

Karl Knechtel
  • 62,466
  • 11
  • 102
  • 153
user443854
  • 7,096
  • 13
  • 48
  • 63

9 Answers9

254

Use itertools.cycle, that's its exact purpose:

from itertools import cycle

lst = ['a', 'b', 'c']

pool = cycle(lst)

for item in pool:
    print item,

Output:

a b c a b c ...

(Loops forever, obviously)


In order to manually advance the iterator and pull values from it one by one, simply call next(pool):

>>> next(pool)
'a'
>>> next(pool)
'b'
Lukas Graf
  • 30,317
  • 8
  • 77
  • 92
  • 4
    You are printing items in a loop. What I want to leave the loop and come back later? (I want to start where I left off). – user443854 May 01 '14 at 21:02
  • 9
    @user443854 use `pool.next()` to get the single next item from the cycle – Jacob Krall May 01 '14 at 21:03
  • What @JacobKrall said ;-) You can "manually" forward any iterator by calling `iterator.next()` on it. That's basically what a `for` loop does for you. – Lukas Graf May 01 '14 at 21:04
  • 5
    @user443854 FWIW this is a much better answer than mine. No reason to go around re-implementing library functions! – Jacob Krall May 01 '14 at 21:05
  • 9
    pool.next() didn't work for me, only next(pool). Probably because of Python 3? – fjsj Aug 21 '15 at 15:23
  • 10
    @fjsj that is correct, on Python 3 you need to use `next(iterator)` (which BTW also works just fine on Python 2.x, and therefore is the canonical form that should be used). See [Is generator.next() visible in python 3.0?](http://stackoverflow.com/questions/1073396/is-generator-next-visible-in-python-3-0) for a more in-depth explanation. Updated my answer accordingly. – Lukas Graf Aug 21 '15 at 18:54
77

The correct answer is to use itertools.cycle. But, let's assume that library function doesn't exist. How would you implement it?

Use a generator:

def circular():
    while True:
        for connection in ['a', 'b', 'c']:
            yield connection

Then, you can either use a for statement to iterate infinitely, or you can call next() to get the single next value from the generator iterator:

connections = circular()
next(connections) # 'a'
next(connections) # 'b'
next(connections) # 'c'
next(connections) # 'a'
next(connections) # 'b'
next(connections) # 'c'
next(connections) # 'a'
#....
Jacob Krall
  • 28,341
  • 6
  • 66
  • 76
  • 1
    Nice! How does it know to start over when the list is exhausted? – user443854 May 01 '14 at 20:56
  • 2
    @user443854 the `while True` means to repeat forever – Jacob Krall May 01 '14 at 20:57
  • 5
    @juanchopanza: Yep; `itertools.cycle` is a better answer. This shows how you could write the same functionality if `itertools` isn't available :) – Jacob Krall May 01 '14 at 21:01
  • Does the simple generator also save a copy of each element like `itertools.cycle` does? Or would the simple generator be a more memory-efficient design? Per the [`cycle` docs](https://docs.python.org/3/library/itertools.html#itertools.cycle): `Note, this member of the toolkit may require significant auxiliary storage (depending on the length of the iterable).` – dthor May 16 '16 at 22:18
  • 2
    @dthor this generator creates a list with three elements and literates over it, then destroys the list and creates a new one, in perpetuity. That documentation for `cycle` implies that the input iterable is converted to `list` before its generator starts, since `iterable` is only "good for one pass over the set of values". – Jacob Krall May 17 '16 at 02:30
  • If you define your list outside the generator, you won't have to recreate the whole list upon creation of the generator, thus saving memory with regard to itertools.cycle (handy when cycling over long/memory-inducing genrators!) – Maxim Jan 20 '18 at 22:42
16

Or you can do like this:

conn = ['a', 'b', 'c', 'd', 'e', 'f']
conn_len = len(conn)
index = 0
while True:
    print(conn[index])
    index = (index + 1) % conn_len

prints a b c d e f a b c... forever

Jabba
  • 19,598
  • 6
  • 52
  • 45
viky.pat
  • 725
  • 9
  • 16
6

you can accomplish this with append(pop()) loop:

l = ['a','b','c','d']
while True:
    print l[0]
    l.append(l.pop(0))

or for i in range() loop:

l = ['a','b','c','d']
ll = len(l)
while True:
    for i in range(ll):
       print l[i]

or simply:

l = ['a','b','c','d']

while True:
    for i in l:
       print i

all of which print:

>>>
a
b
c
d
a
b
c
d
...etc.

of the three I'd be prone to the append(pop()) approach as a function

servers = ['a','b','c','d']

def rotate_servers(servers):
    servers.append(servers.pop(0))
    return servers

while True:
    servers = rotate_servers(servers)
    print servers[0]
Zim
  • 410
  • 4
  • 13
litepresence
  • 3,109
  • 1
  • 27
  • 35
  • 1
    Upvoting this because it helped me with a completely different use case where I simply want to iterate over a list a number of times, each time with the start element advancing one step. My use case is to iterate over the players in a game of poker, advancing the dealer puck one player forward for each round. – Johan Aug 31 '18 at 20:36
  • 1
    Removing an item from the front of a Python list is slow since all elements of the list have to be shifted. The official documentation warns against this (see [link](https://docs.python.org/3/tutorial/datastructures.html#using-lists-as-queues)). Use a [deque](https://docs.python.org/3/library/collections.html#collections.deque) instead! – Thomas Auzinger Feb 09 '22 at 15:21
4

If you wish to cycle n times, implement the ncycles itertools recipe:

from itertools import chain, repeat


def ncycles(iterable, n):
    "Returns the sequence elements n times"
    return chain.from_iterable(repeat(tuple(iterable), n))


list(ncycles(["a", "b", "c"], 3))
# ['a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c']
pylang
  • 40,867
  • 14
  • 129
  • 121
3

You need a custom iterator -- I'll adapt the iterator from this answer.

from itertools import cycle

class ConnectionPool():
    def __init__(self, ...):
        # whatever is appropriate here to initilize
        # your data
        self.pool = cycle([blah, blah, etc])
    def __iter__(self):
        return self
    def __next__(self):
        for connection in self.pool:
            if connection.is_available:  # or however you spell it
                return connection
Community
  • 1
  • 1
Ethan Furman
  • 63,992
  • 20
  • 159
  • 237
3

In order to avoid infinite loop, I have used length of array to iterate only until size of list is double.You can implement your own pre condition .Idea is to avoid infinite loop.

#Implement Circular Linked List
from itertools import cycle
list=[1,2,3,4,5]
lstlength=len(list)*2
print(lstlength)
pool=cycle(list)
i=0
#To avoid infinite loop break when you have iterated twice size of the list
for items in pool:
    print(items)
    if i >lstlength:
        break
    i += 1
user7258708
  • 251
  • 2
  • 7
0
class A(object):
    def __init__(self, l):
        self.strt = 0
        self.end = len(l)
        self.d = l

    def __iter__(self):
        return self

    def __next__(self):
        val = None
        if self.strt>=self.end:
            self.strt=0
        val = self.d[self.strt]
        self.strt += 1
        return val

a= A([8,9,7,66])
print(next(a))
print(next(a))
print(next(a))
print(next(a))
print(next(a))
print(next(a))
print(next(a))
print(next(a))
print(next(a))
print(next(a))
Argus Malware
  • 773
  • 7
  • 19
0

For those, who may be interested. To loop forward or backward starting from given index:

def loop_fwd(arr, index):
  while True:
    arr_index = index % len(arr)
    yield arr_index, arr[arr_index]
    index += 1


def loop_bcw(arr, index):
  while True:
    arr_index = index % len(arr)
    yield arr_index, arr[arr_index]
    index -= 1


forward_it = loop_fwd([1,2,3,4,5], 3)
backward_it = loop_bcw([1,2,3,4,5], 3)

print('forward:')
for i in range(10):
  print(next(forward_it))


print('backward:')
for i in range(10):
  print(next(backward_it))
Dmitriy
  • 486
  • 5
  • 11