Python: Queue.Empty Exception Handling

Question

After a short debate with someone about exception handling in Python - sparked by the handling of a queue object - I thought I'd throw it out there...

METHOD 1:

import Queue

q = Queue.Queue()

try:
    task=q.get(False)
    #Opt 1: Handle task here and call q.task_done()
except Queue.Empty:
    #Handle empty queue here
    pass

#Opt2: Handle task here and call q.task_done()

METHOD 2:

import Queue

q = Queue.Queue()

if q.empty():
    #Handle empty queue here
else:
    task = q.get()
    #Handle task here
    q.task_done()

One argument is that Method 1 is wrong because the queue being empty is not an error, and therefore should not be handled using Queue.Empty exception. Additionally, it could make debugging more difficult when coded this way if you consider that the task handling part could potentially large.

The other argument is that either way is acceptable in Python and that handling the task outside of the try/except could aid debugging if task handling is large, although agreed that this might look uglier than using Method 2.

Opinions?

UPDATE: A little more info after answer 1 came through.... The debate was started after method 1 was using in some multithreaded code. In which case, the code will acquire the lock (from a threading.Lock object) and release it either once the task it returned or Queue.Empty is thrown

UPDATE 2: It was unknown to both of us that the the queue object was thread safe. Looks like try/except is the way to go!

Answer 1

Method 2 is wrong because you are doing an operation in two steps when it could be done in one. In method 2, you check if the queue is empty, and then later (very soon, but still later), try to get the item. What if you have two threads pulling items from the queue? The get() could still fail with an empty queue. What if an item is added to the queue after you checked that it was empty? These are the sort of tiny windows of opportunity where bugs creep in to concurrent code.

Do it in one step, it's by far the better choice.

import Queue

q = Queue.Queue()

try:
    task = q.get(False)
except Queue.Empty:
    # Handle empty queue here
    pass
else:
    # Handle task here and call q.task_done()

Don't get hung up on "exceptions should be errors". Exceptions are simply another channel of communication, use them. Use the "else" clause here to narrow the scope of the exception clause.

Answer 2

If this is multithreaded / multiprocessed code (as is a good reason for using queues anyway), then definitely method 1. Between the q.empty() call and the q.get() call, the Jack of Hearts could have stolen your tarts!

Answer 3

One argument is that Method 1 is wrong because the queue being empty is not an error, and therefore should not be handled using Queue.Empty exception

An exception is not necessarily an "error", it's a general flow control mechanism, and is indeed used that way in a few cases (SysExit, StopIteration etc).

The good question here is: what will be the most common case - empty or non-empty queue. Unless you know for sure, you want to AskBeforeYouLeap, cause it's very probably way cheaper.

Answer 4

Faced the same issue of not finding Queue module; I know this is about a year later, but future readers may find the following model example helpful:

from queue import Queue, Full, Empty
q = Queue(maxsize=3)    # finite queue, maxsize <= 0 -> infinite queue

# enqueue a couple of items
q.put('a')
q.put('b')

# trying to dequeue excessively
for _ in range(4):
    try:
        print(f'dequeued: {q.get(block=False)}')

    except Empty:
        print('empty queue!')

# trying to enqueue excessively
for ch in ['a', 'b', 'c', 'd']:
    try:
        print(f'enqueueing : {ch} ->', end=' ')
        q.put(ch, block=False)
        print('success :)')

    except Full:
        print('full queue :(')

and here is the sample output:

dequeued: a
dequeued: b
empty queue!
empty queue!
enqueueing : a -> success :)
enqueueing : b -> success :)
enqueueing : c -> success :)
enqueueing : d -> full queue :(

Please note, by default, get() and put() are both blocking methods (block=True) and will block the process until a new item is inserted into the queue (get), or an item is consumed to allow room for a new item (put). See here for more details.