Sleep takes 86% of time in rabbitpy consumer?

Asked Jun 27 '14 at 08:23

Active Jun 27 '14 at 08:23

Viewed 201 times

I found a rather interesting problem today. I have a queue with 2K messages in it. The consumer:

import rabbitpy
with rabbitpy.Connection('amqp://guest:guest@localhost:5672/%2f') as conn:
    with conn.channel() as channel:
        queue = rabbitpy.Queue(channel, 'example')
        for message in queue.consume_messages():
            message.ack()

takes 41 seconds to get these messages and ack them. (messages vary from 4kB to 52kB) The publisher, however, took 15 seconds to publish them.

Upon profiling, I found that there is a call to sleep were we spend 86% of the time. This, to my application, is not acceptable. Could someone help me get rid of this sleep? (I'm ok if the CPU cycles are wasted or whatever till a message arrives.)

Zoomed in screenshot The culprit found by the profiler

asked Jun 27 '14 at 08:23

Lelouch Lamperouge

8,171
8
49
60

Bigger image link : http://i.imgur.com/K5ce3fV.png – DhruvPathak Jun 27 '14 at 08:26
Done :). I had assumed that it'd be clickable. – Lelouch Lamperouge Jun 27 '14 at 08:27
Is this python2.x? It might be related to [the "balancing act" problem](http://stackoverflow.com/q/9709022/2096752). – shx2 Jun 27 '14 at 08:36
1

Whoa, what's that! I'm going to try that.. Thanks @shx2 – Lelouch Lamperouge Jun 27 '14 at 08:54
The problem, however, is I am not sure how to proceed with this *creating a quick .so file.....* – Lelouch Lamperouge Jun 27 '14 at 08:56
The first step would be to check if your problem goes away with python3, in order to find out if the two problems are related. – shx2 Jun 27 '14 at 08:57
aye.. but for that, I'd have to compile py3 :( (I'm on AWS).. I'll post back my result.. but idk if you'd be available – Lelouch Lamperouge Jun 27 '14 at 08:58
the one in the question is definitely an SSCCE. Also, with some options (shots in the dark) http://paste.ubuntu.com/7710186/ – Lelouch Lamperouge Jun 27 '14 at 09:14
Interesting thing is.. pypy2.7 did it in 15 seconds, while py3 still took ~32 seconds. – Lelouch Lamperouge Jun 27 '14 at 09:21
1

@shx2, pypy did it. It works like magic. I'll profile the code and post it as an asnwer as soon as time permits. Sincere thanks! – Lelouch Lamperouge Jun 27 '14 at 20:20

Sleep takes 86% of time in rabbitpy consumer?

0 Answers0