12

I came across this interesting statement in "Caveats" section of the documentation for the thread module today:

Not all built-in functions that may block waiting for I/O allow other threads to run. (The most popular ones (time.sleep(), file.read(), select.select()) work as expected.)

Pretty much everywhere else I've ever seen Python threads discussed, there has always been an assumption that all built-in functions which do I/O will release the GIL, meaning other threads can run while the function blocks. As far as I knew, the only risk of an I/O operation blocking other threads would be if it was being made to a buggy C-extension that neglected to release the GIL.

So, is this statement from the thread docs actually true? Are there any built-in, blocking I/O operations that don't release the GIL? I haven't been able to find any specific examples thus far.

dano
  • 91,354
  • 19
  • 222
  • 219
  • Maybe it's implementation specific. Maybe CPython has no non-GIL-releasing I/O functions, but other versions are allowed to have them, as long as sleep/read/select remain GIL-releasing. – Kevin Jul 18 '14 at 18:30
  • Yes I believe @Kevin is correct. It just means that developers of extensions are not required to release the GIL, its just good practise inside of the extension to do so. The developer must know that their function is purely waiting on I/O and program the GIL in a specific way. Thus, it is not guaranteed. – beiller Jul 18 '14 at 18:31
  • @beiller The document specifically states "built-in functions", though, which suggests to me that it's not factoring in C-extensions here. – dano Jul 18 '14 at 18:32
  • It just sounds like its an optional optimization, even for built-ins to release the GIL. – beiller Jul 18 '14 at 18:33
  • @beiller Well, according to that statement in the docs, you're right. I'm just curious if there's actually a built-in function that doesn't do it, or if they just "reserve the right" to not do it. – dano Jul 18 '14 at 18:34
  • Yeah that's why I can't answer the question. I know of no built in functions that do not release the lock on I/O. I would program without worrying about it. Perhaps out dated built-ins or esoteric functions may have this issue. – beiller Jul 18 '14 at 18:36
  • 1
    That statement appears to be ancient: I can find it in the SVN history at least as far back as January 1994, in a commit from Guido with message "Restructured library documentation". I'm having trouble tracking it back further than that. – Mark Dickinson Jul 18 '14 at 19:26
  • @MarkDickinson Ah, very interesting. I had wondered if this was just a case of the docs being out of date, especially since the `thread` module is rarely used in modern Python. – dano Jul 18 '14 at 19:29
  • For reference, [here's](https://github.com/python/cpython/commit/5358a5d5edc5fa2c86e92ee752bbdd8ca2c19f2e) the commit in Python's GitHub mirror. – Mark Dickinson Jul 18 '14 at 19:40
  • @MarkDickinson Thanks. Should a bug be filed against the documentation? Or maybe a post on the mailing list? – dano Jul 18 '14 at 19:42
  • Yes, I'd say a bug report seems like a good idea. – Mark Dickinson Jul 18 '14 at 19:44
  • 1
    @MarkDickinson Thanks. Created [issue22006](http://bugs.python.org/issue22006). – dano Jul 18 '14 at 19:54

1 Answers1

11

Here's the official word from Guido van Rossum on this one:

Not in my wildest dreams could I have expected that that claim would still be in the docs 20 years later. :-) Please get rid of it.

So the answer to my question is "No".

dano
  • 91,354
  • 19
  • 222
  • 219