11

I am implementing Python Queue with Thread. I need a way to get the total count of items currently in the queue (which I can get with queue.qsize()) and the count of unfinished tasks. Basically I need a count of all items that are being processed/need to be processed. The Python Queue documentation mentions the following:

The count of unfinished tasks goes up whenever an item is added to the queue. The count goes down whenever a consumer thread calls task_done() to indicate that the item was retrieved and all work on it is complete. When the count of unfinished tasks drops to zero, join() unblocks.

But it offers no insight as to how to access that count. Thanks in advance!

Mocking
  • 1,764
  • 2
  • 20
  • 36
  • 1
    Well this: http://stackoverflow.com/questions/11786530/can-python-threads-access-variables-in-the-namespace is an alternative but I feel like there has to be something built into Queue... – Mocking Apr 16 '16 at 00:03

1 Answers1

16

I'm surprised it is not mentioned in the documentation but Queue objects have a unfinished_tasks attribute:

>>> x = queue.Queue()
>>> x.unfinished_tasks
0
>>> x.put("test")
>>> x.unfinished_tasks
1

Since it is not mentioned in the documentation it isn't guaranteed to be the same attribute updating to newer versions (although I somewhat doubt it) and particularly I wouldn't trust this to be thread-safe (since any internal use would already be inside a lock acquire so external access likely doesn't)

Tadhg McDonald-Jensen
  • 20,699
  • 5
  • 35
  • 59
  • 1
    You sir or madam, are the best. – Mocking Apr 16 '16 at 00:20
  • 2
    Yeah, but... can it be considered part of the public API of the Queue class or an implementation detail? – Alvaro Gutierrez Perez Oct 24 '17 at 03:48
  • 2
    @AlvaroGutierrezPerez shoot I guess not being in the documentation would make it an implementation detail. Well, that is disappointing. – Tadhg McDonald-Jensen Oct 24 '17 at 19:30
  • 3
    Weather things are in the documentation or not does denote weather their part of the API. Documentation is wrong all the time. Python clearly [sates](https://docs.python.org/3/tutorial/classes.html#tut-private) that variables starting with '\_' or '__' are private variables, so anything else should be considered a public part of the API. – Jamie Marshall Apr 23 '19 at 20:27