21

Within Linux, there is a file, /sys/kernel/debug/tracing/trace_pipe, which as the name says, is a pipe. So, let's say I want to read the first 50 bytes from it using Python - and I run the following code:

$sudo python -c 'f=open("/sys/kernel/debug/tracing/trace_pipe","r"); print f; print f.read(50); f.close()<br>
<open file '/sys/kernel/debug/tracing/trace_pipe', mode 'r' at 0xb7757e90>

We can see that opening the file goes fast ( if we have the superuser permissions ) - however, if the trace_pipe file is empty at that moment, it will simply block ( and even if there is content, the content will be dumped until there is no more, and then again the file will block ). Then I have to press Ctrl-C to interrupt the Python script with a KeyboardInterrupt...

How can I have Python 2.7 do a read with timeout?

That is, I want to instruct Python to "try read 50 bytes from this file; if you don't succeed after one second, give up and return"?

user3666197
  • 1
  • 6
  • 50
  • 92
sdaau
  • 36,975
  • 46
  • 198
  • 278

3 Answers3

28

Use

os.read(f.fileno(), 50)

instead. That does not wait until the specified amount of bytes has been read but returns when it has read anything (at most the specified amount of bytes).

This does not solve your issue in case you've got nothing to read from that pipe. In that case you should use select from the module select to test whether there is something to read.

EDIT:

Testing for empty input with select:

import select
r, w, e = select.select([ f ], [], [], 0)
if f in r:
  print os.read(f.fileno(), 50)
else:
  print "nothing available!"  # or just ignore that case
Alfe
  • 56,346
  • 20
  • 107
  • 159
  • Many thanks for that, @Alfe - I forgot about `os.read` ... However, I do want to primarily handle the case when I have nothing to read! Thanks for noting [`select`](http://docs.python.org/2/library/select.html) - but in my case, what is `rlist`, `wlist`, or `xlist`? What should I expect if I have nothing to read? Does `select` tell me total bytes available when I use it, or just "at least 1 byte is available"? – sdaau Jan 29 '14 at 11:31
  • 2
    See my EDIT about `select` on how to use it. `select` returns subsets of the sets of streams you hand to it. Your case is simplistic, you only have one stream you only want to read from, so leave everything else empty. Give a timeout of 0, so it does never _wait_ for anything, and then test whether there is something readable from `f`. If so, at least one byte can be read. Then read at least that one byte and at most 50 bytes (as you specified) from that stream. – Alfe Jan 29 '14 at 11:37
  • 1
    Ah, one thing you should be aware of is the fact that normally a pipe cannot be written to unless there is a reader reading from it. So you will _never_ have anything waiting for being read in that pipe. Maybe you should use a timeout a little larger than 0 ;-) – Alfe Jan 29 '14 at 11:44
  • Many thanks for the edit @Alfe - I have added a post below which is converted to my use case... – sdaau Jan 29 '14 at 11:54
  • @Alfe this is brilliant, you just ended my multi-hour way to get a grip on interactive, non-blocking terminal output from a piped bash command. Thanks!! – fr_andres Jun 28 '20 at 18:21
  • race condition can occur between `select` and `read`. if a different process/thread reads the fd empty, read will block ([example](https://github.com/stefanb2/ninja/blob/15bc8f783b63e9bf91080d9b9d6c83468f012f97/src/tokenpool-gnu-make-posix.cc#L157)) -> wrap the read in a timeout – milahu Jun 30 '22 at 16:17
  • @milahu Thank you for the remark, you are right. Unfortunately the `os.read()` doesn’t offer any timeout handling (which it should). Wrapping a simple `read()` into a monstrosity like the C solution you linked to is clearly against the basic principles of Python (clarity, readability). So let’s just say this solution only works if no other processes read from this file descriptor (which is the case for many scenarios anyway). – Alfe Jul 04 '22 at 11:24
  • 1
    ps: i ported tokenpool-gnu-make-posix.cc to [python](https://github.com/milahu/gnumake-tokenpool/blob/main/py). easier than [javascript](https://stackoverflow.com/questions/20808126/how-to-timeout-an-fs-read-in-node-js) – milahu Jul 04 '22 at 14:03
15
f = os.open("/sys/kernel/debug/tracing/trace_pipe", os.O_RDONLY|os.O_NONBLOCK)

Should prevent blocking (works in Unix only).. No need for select here..

GabiMe
  • 18,105
  • 28
  • 76
  • 113
  • 3
    Many thanks for that, @GabiMe ; wasn't aware that `os.open` offers platform specific options! I was thinking of moving the accept here, but I'll keep in on the first answer, because with this approach, one also needs to be aware of `fdopen`, and also check for exceptions if there is nothing in the file (see the edit in my post below). Thanks again - cheers! – sdaau Jan 29 '14 at 12:08
3

Just adding this as note, for better formatting:

@Alfe's answer in my case:

$ sudo python -c 'import os, select; 
f=open("/sys/kernel/debug/tracing/trace_pipe","r"); print f; 
rrdy, wrdy, xrdy = select.select([f], [], [], 1); print rrdy, wrdy, xrdy ; 
timeout= "timed out" if (rrdy==[]) else "" ; 
print timeout; 
print os.read(f.fileno(), 50) if timeout=="" else ""; 
f.close() '

If there is something in the file, I get response like:

<open file '/sys/kernel/debug/tracing/trace_pipe', mode 'r' at 0xb76f0e90>
[<open file '/sys/kernel/debug/tracing/trace_pipe', mode 'r' at 0xb76f0e90>] [] []

            Xorg-1033  [001] 12570.075859: <user s

If there is nothing in the file, I get:

<open file '/sys/kernel/debug/tracing/trace_pipe', mode 'r' at 0xb7831e90>
[] [] []
timed out

Note that the select documentation isn't explicit that the timeout parameter is in seconds - but that floating point values (e.g. 0.5) also work.

@GabiMe's answer:

$ sudo python -c 'import os; 
filno = os.open("/sys/kernel/debug/tracing/trace_pipe", os.O_RDONLY|os.O_NONBLOCK); 
f=os.fdopen(filno, "r"); print f; 
print "A", f.read(50); 
print "B", os.read(f.fileno(), 50); 
f.close() '

If there is something in the file, I get response like:

<open file '<fdopen>', mode 'r' at 0xb77b6e90>
A             bash-13777 [000] 13694.404519: sys_exi
B            Timer-31065 [001] 13694.404830: sys_exi

If there is nothing in the file, I get:

<open file '<fdopen>', mode 'r' at 0xb77c1e90>
A
Traceback (most recent call last):
  File "<string>", line 1, in <module>
IOError: [Errno 11] Resource temporarily unavailable

... so one must run this in a try block, to catch the IOError, if there is nothing in the file... (both os.read and f.read will raise this exception)

sdaau
  • 36,975
  • 46
  • 198
  • 278
  • From the docu you linked to about `select`: »The optional timeout argument specifies a time-out as a floating point number in seconds.« Why do you think it doesn't state that the `timeout` parameter is in seconds? – Alfe Jun 19 '14 at 07:37