Keyboard interruptable blocking queue in Python

Question

It seems

import Queue

Queue.Queue().get(timeout=10)

is keyboard interruptible (ctrl-c) whereas

import Queue

Queue.Queue().get()

is not. I could always create a loop;

import Queue
q = Queue()

while True:
    try:
        q.get(timeout=1000)
    except Queue.Empty:
        pass

but this seems like a strange thing to do.

So, is there a way of getting an indefinitely waiting but keyboard interruptible Queue.get()?

This is [Bug 1360](http://bugs.python.org/issue1360) which was closed as "won't fix". The suggested workaround is to always specify a timeout if you need interruption. — dimo414, Jun 18 '15 at 18:24

score 6 · Accepted Answer · answered Oct 17 '08 at 17:01

6

Queue objects have this behavior because they lock using Condition objects form the threading module. So your solution is really the only way to go.

However, if you really want a Queue method that does this, you can monkeypatch the Queue class. For example:

def interruptable_get(self):
    while True:
        try:
            return self.get(timeout=1000)
        except Queue.Empty:
            pass
Queue.interruptable_get = interruptable_get

This would let you say

q.interruptable_get()

instead of

interruptable_get(q)

although monkeypatching is generally discouraged by the Python community in cases such as these, since a regular function seems just as good.

answered Oct 17 '08 at 17:01

Eli Courtwright

186,300
67
213
256

Ah, I see, a bad case of leaky abstractions*. Ahwell. * http://www.joelonsoftware.com/articles/LeakyAbstractions.html – Henrik Gustafsson Oct 17 '08 at 17:18
Why not subclass Queue, rather than monkey patching it? – Alex Coventry Oct 17 '08 at 18:06
Monkeypatching is better when you're receiving Queue objects that were instantiated and returned by some third-party code that you can't change, or possibly by some co-worker's code that you also can't change. However, perhaps I should have used subclassing since that's probably not very common. – Eli Courtwright Oct 17 '08 at 19:25
"`Queue` objects have this behavior because they lock using `Condition` objects form the `threading` module." I am not sure why this is the case. Can someone explain or point me to a resource that explains this behavior? – joshreesjones Jun 16 '16 at 19:26
@joshreesjones, dimo414's comment is correct. See http://bugs.python.org/issue1360. – bennlich Nov 14 '16 at 21:29

score 4 · Answer 2 · answered Oct 19 '08 at 17:40

This may not apply to your use case at all. But I've successfully used this pattern in several cases: (sketchy and likely buggy, but you get the point).

STOP = object()

def consumer(q):
    while True:
        x = q.get()
        if x is STOP:
            return
        consume(x)

def main()
    q = Queue()
    c=threading.Thread(target=consumer,args=[q])

    try:
        run_producer(q)
    except KeybordInterrupt:
        q.enqueue(STOP)
    c.join()

Keyboard interruptable blocking queue in Python

2 Answers2

Linked