How to iterate Queue.Queue items in Python?

Question

Does anyone know a pythonic way of iterating over the elements of a Queue.Queue without removing them from the Queue. I have a producer/consumer-type program where items to be processed are passed by using a Queue.Queue, and I want to be able to print what the remaining items are. Any ideas?

score 52 · Answer 1 · edited Oct 12 '14 at 11:27

52

You can loop over a copy of the underlying data store:

for elem in list(q.queue)

Eventhough this bypasses the locks for Queue objects, the list copy is an atomic operation and it should work out fine.

If you want to keep the locks, why not pull all the tasks out of the queue, make your list copy, and then put them back.

mycopy = []
while True:
     try:
         elem = q.get(block=False)
     except Empty:
         break
     else:
         mycopy.append(elem)
for elem in mycopy:
    q.put(elem)
for elem in mycopy:
    # do something with the elements

edited Oct 12 '14 at 11:27

sebdelsol

1,045
8
15

answered Nov 19 '11 at 20:06

Raymond Hettinger

216,523
63
388
485

4

`for elem in list(q.queue)` results in `TypeError: 'Queue' object is not iterable` in Python 3. Perhaps I am doing something wrong? – Jonathan Komar Jan 25 '17 at 15:22
4

@macmadness86 It looks like you have another layer with "q" being code object that has a "queue" attribute that holds a Queue object. Try this: ``for elem in list(q.queue.queue)``. – Raymond Hettinger Jan 26 '17 at 00:58
1

Roger that. Will comply. Thanks for the tip. (this message is scheduled for deletion) – Jonathan Komar Jan 26 '17 at 06:34

Omer Dagan · Answer 2 · 2018-04-08T09:07:45.343

24

Listing queue elements without consuming them:

>>> from Queue import Queue
>>> q = Queue()
>>> q.put(1)
>>> q.put(2)
>>> q.put(3)
>>> print list(q.queue)
[1, 2, 3]

After operation, you can still process them:

>>> q.get()
1
>>> print list(q.queue)
[2, 3]

edited Apr 08 '18 at 09:07

answered Dec 19 '17 at 12:39

Omer Dagan

14,868
16
44
60

score 9 · Answer 3 · answered Mar 04 '16 at 16:00

You can subclass queue.Queue to achieve this in a thread-safe way:

import queue


class ImprovedQueue(queue.Queue):
    def to_list(self):
        """
        Returns a copy of all items in the queue without removing them.
        """

        with self.mutex:
            return list(self.queue)

score 1 · Answer 4 · answered Dec 06 '21 at 14:49

You can convert the deque into a list before printing the elements so that you can easily iterate through it.

from collections import deque

d = deque([7,9,3,5])

d.append(2)
d.appendleft(1)
d.append(10)
d.pop()

for elem in list(d):
    print(elem, end=" ")

#Output: 1 7 9 3 5 2

How to iterate Queue.Queue items in Python?

4 Answers4

Linked