1

Context

I'm using windows 7 on my computer(the player) and linux(debian) on my college computer(the streamer), which I control using ssh.
I was trying to simulate a constant byte rate of a microphone from reading a wave file, as if someone was talking. The problem was that the byte rate was below the target.

Choosing a 32KB/s rate, and 0.020 seconds of capture time.
I implemented the simulated microphone using time.sleep to produce give each chunk of data each 0.020 seconds. But the rate obtained was around 27KB/s, not 32KB/s

The problem

I decided to test how much precise time.sleep was on the linux machine, using ideas by reading this question.

I did 2 kind of tests. 1) busy sleep 2) normal sleep

In average, from the samples I got, it shows that the linux machine sleep resolution is 4ms. While on windows its less/equal to 1ms.

Questions

  1. What can possibly be limiting the sleep resolution on the linux machine?
  2. (On linux) Why does busy sleeping has the same resolution as time.sleep?
  3. How could I successfully simulate a microphone from reading a wave file?

Code

import time

def busy_sleep(t):
    s=time.time()
    while time.time() - s < t:
        pass
    e=time.time()
    return e-s

def normal_sleep(t):
    s=time.time()
    time.sleep(t)
    e=time.time()
    return e-s

def test(fun):
    f = lambda x: sum(fun(x) for d in range(10))/10
    print("0.100:{}".format(f(0.100)))
    print("0.050:{}".format(f(0.050)))
    print("0.025:{}".format(f(0.025)))
    print("0.010:{}".format(f(0.010)))
    print("0.009:{}".format(f(0.010)))
    print("0.008:{}".format(f(0.008)))
    print("0.007:{}".format(f(0.007)))
    print("0.006:{}".format(f(0.006)))
    print("0.005:{}".format(f(0.005)))
    print("0.004:{}".format(f(0.004)))
    print("0.003:{}".format(f(0.003)))
    print("0.002:{}".format(f(0.002)))
    print("0.001:{}".format(f(0.001)))

if __name__=="__main__":
    print("Testing busy_sleep:")
    test(busy_sleep)
    print("Testing normal_sleep:")
    test(normal_sleep)

Results

"""
Debian
Testing busy_sleep:
0.100:0.10223722934722901
0.050:0.051996989250183104
0.025:0.027996940612792967
0.020:0.02207831859588623
0.010:0.011997451782226562
0.009:0.011997222900390625
0.008:0.009998440742492676
0.007:0.007997279167175292
0.006:0.0079974365234375
0.005:0.007997465133666993
0.004:0.005918483734130859
0.003:0.003997836112976074
0.002:0.0039977550506591795
0.001:0.003997611999511719
Testing normal_sleep:
0.100:0.1020797061920166
0.050:0.051999988555908205
0.025:0.028000001907348634
0.020:0.02192000865936279
0.010:0.011999979019165039
0.009:0.012000055313110351
0.008:0.010639991760253906
0.007:0.008000001907348633
0.006:0.00799997329711914
0.005:0.008000059127807617
0.004:0.006159958839416504
0.003:0.004000000953674317
0.002:0.00399998664855957
0.001:0.004000091552734375

$ uname -a
Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.57-3+deb7u2 x86_64 GNU/Linux
"""

"""
Windows 7

Testing busy_sleep:
0.100:0.10000572204589844
0.050:0.05000288486480713
0.025:0.0250014066696167
0.010:0.010500597953796388
0.009:0.010500597953796388
0.008:0.008000493049621582
0.007:0.00740041732788086
0.006:0.006400299072265625
0.005:0.005400300025939942
0.004:0.004700303077697754
0.003:0.003200197219848633
0.002:0.002700185775756836
0.001:0.0016000032424926759
Testing normal_sleep:
0.100:0.10000579357147217
0.050:0.0500028133392334
0.025:0.02500150203704834
0.010:0.01000049114227295
0.009:0.0100006103515625
0.008:0.008000493049621582
0.007:0.007000398635864257
0.006:0.006000304222106934
0.005:0.00500030517578125
0.004:0.0040001869201660155
0.003:0.0030002117156982424
0.002:0.0020000934600830078
0.001:0.0010000944137573243
"""

Real code

import os import wave import sys import io import time

FORMAT = 8 #get_format_from_width(2)
NCHANNELS = 1
FRAMERATE = 16000 # samples per second
SAMPWIDTH = 2 # bytes in a sample
BYTE_RATE = FRAMERATE*SAMPWIDTH
CHUNK_DURATION = 0.020
CHUNK_BYTES = int(CHUNK_DURATION*BYTE_RATE)

class StreamSimulator:
    def __init__(self):
        wf = wave.open("Kalimba.wav","rb")
        buf = io.BytesIO()
        buf.write(wf.readframes(wf.getnframes()))
        wf.close()
        buf.seek(0)
        self.buf = buf
        self.step = time.time()

    def delay(self):
        #delay
        delta = time.time() - self.step 
        self.step=time.time()
        delay = CHUNK_DURATION - delta      
        if delay > 0.001:
            time.sleep(delay)


    def read(self):
        buf = self.buf  
        data = buf.read(CHUNK_BYTES)
        if len(data) == 0:
            buf.seek(0)
            data = buf.read(CHUNK_BYTES)

        self.delay()
        return data

    def close(self):
        self.buf.close()

class DynamicPainter:
    def __init__(self):
        self.l=0

    def paint(self,obj):        
        str1=str(obj)
        l1=len(str1)
        bs="\b"*self.l
        clean=" "*self.l
        total = bs+clean+bs+str1
        sys.stdout.write(total)
        sys.stdout.flush()
        self.l=l1

if __name__=="__main__":
    painter = DynamicPainter()
    stream = StreamSimulator()
    produced = 0
    how_many = 0
    painted = time.time()
    while True:
        while time.time()-painted < 1:
            d = stream.read()
            produced += len(d)
            how_many += 1
        producing_speed = int(produced/(time.time()-painted))       
        painter.paint("Producing speed: {} how many: {}".format(producing_speed,how_many))
        produced=0
        how_many=0
        painted = time.time()

Edit

Changed "Real Code" , added measure of time including sleeping time.
But now I have the DOUBLE Byte Rate: Producing speed: 63996 how many: 100
This got me so MUCH confused. I have tried with different byterates and it ends up being the double everytime.

Conclusion

Thanks to @J.F.Sebastian and his code, I came to learned that:

  • Its better to use use a deadline as a time reference than create to a new reference each loop
  • Using a deadline "amortizes" the imprecision of time.sleep, oscilating a bit around the desired bitrate but resulting in an correct(and much more constant) average.
  • You only need to you use time.time() once, that means less calculations imprecisions.

As a result, I get a constant 32000 B/s some times oscilating to 31999 and very rarely to 31745
Now I can hear the music without any lag or jitter!

Final Code

def read(self):
    buf = self.buf  
    data = buf.read(CHUNK_BYTES)
    if len(data) == 0:
        buf.seek(0)
        data = buf.read(CHUNK_BYTES)

    self.deadline += CHUNK_DURATION 
    delay = self.deadline - time.time()
    if delay > 0:
        time.sleep(delay)
    return data
Community
  • 1
  • 1
Rui Botelho
  • 752
  • 1
  • 5
  • 18
  • 1
    "*I was trying to simulate a constant byte rate*" - Tell us more abot that. *That* seems to be where the problem is, not in `time.sleep()`. – Robᵩ Oct 27 '14 at 19:56
  • @Robᵩ Added the real code to the question – Rui Botelho Oct 27 '14 at 20:08
  • 1
    here's a [script that generates data at constant rate](https://gist.github.com/zed/9cb41b2cfe615a7be3e9). – jfs Oct 27 '14 at 21:33
  • @J.F.Sebastian thank you for mentioning monotonic, didnt know about that! But unfortunately now I have another problem, as you cann see the main post – Rui Botelho Oct 27 '14 at 22:11
  • `time.monotonic()` does *not* change the result here unless you system time is set back; you could use `time.time()`. Run the script, it works like metronome. What else do you need? – jfs Oct 27 '14 at 22:14
  • @J.F.Sebastian Yes, I know it doesnt! But somehow the code is generating the double rate of what is supposed to. With either functions. – Rui Botelho Oct 27 '14 at 22:22
  • to understand the code, start with the simple code (like the one that I've provided), test it to make sure it works, and modify it step by step, verifying each step, until it does what you need. – jfs Oct 27 '14 at 22:35
  • You should post everything after the word "Conclusion" as an answer. [Note: I use an integer deadline (infinite precision)](https://gist.github.com/zed/9cb41b2cfe615a7be3e9#file-constant-bit-rate-py). If `CHUNK_DURATION` is a float than `self.deadline` may lose precision after a while. [You could use `timer() % CHUNK_DURATION` to keep it within range](https://gist.github.com/zed/9cb41b2cfe615a7be3e9#file-constant-bit-rate-modulo-py). Also use `timeit.default_timer()` to get consistent behavior on multiple platforms. – jfs Oct 28 '14 at 11:00
  • @J.F.Sebastian Brilliant idea to use the remainder! I will post my conclusio as the answer. I wasnt sure I could!! – Rui Botelho Oct 28 '14 at 11:44
  • @RuiBotelho: the idea is not mine. I might have got it from [What is the best way to repeatedly execute a function every x seconds in Python?](http://stackoverflow.com/a/25251804/4279). – jfs Oct 28 '14 at 16:45

2 Answers2

1

Conclusion

Thanks to @J.F.Sebastian and his code, I came to learned that:

  • Its better to use use a deadline as a time reference than create to a new reference each loop
  • Using a deadline "amortizes" the imprecision of time.sleep, oscilating a bit around the desired bitrate but resulting in an correct(and much more constant) average.
  • You only need to you use time.time() once, that means less calculations imprecisions.

As a result, I get a constant 32000 B/s some times oscilating to 31999 and very rarely to 31745
Now I can hear the music without any lag or jitter!

I tried using @J.F.Sebastian implentation using only the % operator to sleep the remainder, but the KB/s oscilate strangelly, so I decided to keep the deadline implementation, which suffers imprecision by keeping adding a float value. Nevertheless, the overall result is enough for my needs.
Thank you everyone.

Final Code

def read(self):
    self.deadline += 0.020  
    delay = self.deadline - time.perf_counter()
    if delay > 0:
        time.sleep(delay)
    return self._read()
Rui Botelho
  • 752
  • 1
  • 5
  • 18
0

As you can read in the question you linked, there are no guarantees for sleep and it can vary widely depending on the OS. But 4 ms resolution seems sufficient if you want to send data every 20 ms. There should be no need to improve your sleep() accuracy.

On Debian, at around 0.02 seconds input, your computer sleeps around 12 / 10 of the time you ask it to. 10 / 12 * 32 is 26.6, so it makes sense if you only receive 27 KB/s.

Instead of sleeping 0.02 seconds, use an adaptive sleeping time. Measure how long the last iteration took (sleeping + sending the data), and shorten your sleeping time to make the whole iteration take 0.02 seconds.

w-m
  • 10,772
  • 1
  • 42
  • 49
  • I forgot to include the 0.02 measure! Added it just now. But I can reach the same conclusion that you said. Besides shortening the sleeping time, could I also try to predict the next sleeping time? like using average or some other statistical method? – Rui Botelho Oct 27 '14 at 20:26
  • I don't think you need to do something fancy, the delays shoulde even out. Just measure how long your read() took, subtract your sleep time and you'll get your overhead. Compute your next sleeping time as 0.02 - overhead. If that is not accurate enough, keep track of the global time and work against the drift. – w-m Oct 27 '14 at 21:06
  • Yes, I tried the simple approach but now I have the double KB/s... I edited the question "real code". I cannot see the reason... – Rui Botelho Oct 27 '14 at 22:13
  • You forgot the part where you subtract the time you told python to sleep. Add `delta -= self.last_delay` and you'll get the correct rate. – w-m Oct 27 '14 at 22:43
  • Sorry, I found that J.F. Sebastian solution worked better! Check my question conclusion. otherwise, thank you! – Rui Botelho Oct 27 '14 at 23:39
  • Great! Your solution is very concise, and you only need to store the deadline, instead of the last time and the last delay. The solutions should be identical in their accuracy, though. – w-m Oct 28 '14 at 00:39