56

I've got a thread that's polling a piece of hardware.

while not hardware_is_ready():
    pass
process_data_from_hardware()

But there are other threads (and processes!) that might have things to do. If so, I don't want to burn up cpu checking the hardware every other instruction. It's been a while since I've dealt with threading, and when I did it wasn't Python, but I believe most threading libraries have a yield function or something that allows a thread to tell the scheduler "Give the other threads a chance."

while not hardware_is_ready():
    threading.yield()          # This function doesn't exist.
process_data_from_hardware()

But I can't find any reference to something like this in the threading documentation. Python does have a yield statement, but I'm pretty sure that's something else entirely (to do with generators).

What's the correct thing to do here?

Tom Future
  • 1,900
  • 2
  • 15
  • 15

4 Answers4

91

time.sleep(0) is sufficient to yield control -- no need to use a positive epsilon. Indeed, time.sleep(0) MEANS "yield to whatever other thread may be ready".

Alex Martelli
  • 854,459
  • 170
  • 1,222
  • 1,395
  • 6
    Is this documented somewhere? I thought this might be true -- I've seen it in other languages -- but I couldn't find a mention of it in the Python docs. – Tom Future Apr 27 '09 at 03:34
  • 5
    for example, http://docs.python.org/library/sched.html "delayfunc will also be called with the argument 0 after each event is run to allow other threads an opportunity to run in multi-threaded applications." (delayfunc is usually time.sleep here). I agree it would be nicer if it was documented more directly in module time;-) – Alex Martelli Apr 27 '09 at 17:39
  • 5
    Windows users: setting to 0 didn't work for me, time.sleep( 0.0001 ) did. – driedler Mar 29 '18 at 21:52
  • This is not working on Windows 7 with python 3.6.4 for me. – waszil Dec 03 '18 at 14:50
14

Read up on the Global Interpreter Lock (GIL).

For example: http://jessenoller.com/2009/02/01/python-threads-and-the-global-interpreter-lock/

Also: http://www.pyzine.com/Issue001/Section_Articles/article_ThreadingGlobalInterpreter.html

Do this in your code if you must do Busy Waiting (e.g. polling a device).

time.sleep( 0.0001 )

This will yield to the thread scheduler.

Also, I collected some notes and references in http://homepage.mac.com/s_lott/iblog/architecture/C551260341/E20081031204203/index.html

S.Lott
  • 384,516
  • 81
  • 508
  • 779
  • 2
    I'm no expert, but I've read a bit on the GIL. Could you give me a hint what in particular relates to this question? I'll go with `time.sleep(epsilon)` for now, but it's kind of unsatisfying, isn't it? There isn't any particular interval I want to sleep for. – Tom Future Apr 24 '09 at 22:45
  • Its the only solution if your hardware cannot send interrupts. – Unknown Apr 25 '09 at 01:58
  • Without using `time.sleep()` how does Python perform pre-emptive yielding? On what conditions does a yield occur? – CMCDragonkai Oct 10 '16 at 13:23
  • 4
    All three links seem broken. – trincot Aug 03 '19 at 16:38
6

Why the yield in time.sleep(0) might not be enough:

I had a similar problem and ultimately time.sleep(0) did not work in all cases whereas time.sleep(0.0001) worked. In my case I use the standard vanilla cpython - which can be found at python.org. Looking at the implementation of the sleep function in the C code there is a tiny difference between a call of the sleep(0) and sleep(something_other_than_null).

Looking at pysleep in timemodule.c you will see that in the first case

if (ul_millis == 0 || !_PyOS_IsMainThread()) {
  Py_BEGIN_ALLOW_THREADS
  Sleep(ul_millis);
  Py_END_ALLOW_THREADS
  break;
}

is called. The "allow threads" releases the GIL, Sleep(0) in that case is equivalent to std::this_thread::yield(), after that the GIL is claimed again. In the code that deals with a non-zero value, apart from the actual sleep/wait there is an additional call to PyErr_CheckSignals - which as stated in its documentation "checks whether a signal has been sent to the processes and if so, invokes the corresponding signal handler". That would explain why in some cases when working with hardware a time.sleep(0) is not enough. So this might be a slip on cpython implementation side. And that is why time.sleep(0.0001) magically works.

michael_s
  • 2,515
  • 18
  • 24
4

If you're doing this on *nix, you might find the select library useful. Kamaela also has a few components you may find useful, but it may require a bit of a paradigm change.

Jason Baker
  • 192,085
  • 135
  • 376
  • 510