Why does reading from /sys/bus/... take so much time?

Question

I'm working on a project which requires sensor data from a temperature sensor. While accessing the file using open() and then read(), we found that it took too long. We have isolated problem to read() taking the most time (approximately 1 second). Is there a faster alternative to read() or am I using it incorrectly? Code:

import time, os, socket

#External thermometer address: 28-031897792ede

os.system('modprobe w1-gpio')
os.system('modprobe w1-therm')

temp_sensor = '/sys/bus/w1/devices/28-031897792ede/w1_slave'

def temp_raw():
    f = open(temp_sensor, 'r')
    lines = f.readlines()
    f.close()
    return lines

def read_temp():

    lines = temp_raw()
    while lines[0].strip()[-3:] != 'YES':
        lines = temp_raw()

    temp_output = lines[1].find('t=')

    if temp_output != -1:
        temp_string = lines [1].strip()[temp_output+2:]
        temp_c = float(temp_string) / 1000.0
        return round(temp_c, 1)

while True:
    temp_raw()

`lines = f.read()` be careful you probably mean `lines = f.readlines()` — Jean-François Fabre, May 08 '19 at 21:44
that loop is probably infinite: `while lines[0].strip()[-3:] != 'YES': lines = temp_raw()` since you're reading the file each time so if it matches once it matches forever — Jean-François Fabre, May 08 '19 at 21:45
https://stackoverflow.com/questions/30294146/python-fastest-way-to-process-large-file — Matthew Barlowe, May 08 '19 at 21:46
@Jean-FrançoisFabre The file is a simple text file which only has two lines. Testing temp_raw() for times shows that readlines() nearly takes 1 second, but not the other function as far as I can tell. — Aditya Kondapuram, May 08 '19 at 21:59
There are several problems with this in function and efficiency. Most of all, I'm not sure what you're trying to do, since your main program ignores `read_temp` entirely. You have an inefficient infinite loop that reads the file and quits without any processing. Please describe what you *are* trying to do with this. I'm not sure why you've bypassed the examples of file-handling tutorials for this implementation. — Prune, May 08 '19 at 22:13
@Prune this was the example code for the temperature sensor. It did use read_temp() to normalize the sensor data to a value which makes sense. To troubleshoot why it was taking so long, we tested each part and found that the issue was in temp_raw() and specifically happened on readlines(). — Aditya Kondapuram, May 08 '19 at 22:27
Please correct your code. There is no reference to `readlines` in what you posted. Again, what is the goal of this code? Even if you do call `read_temp`, all that function seems to do is to find a single line in the file and extract a particular `float` field. To this purpose, it reads the *entire* file. That functionality would probably be done better with an OS command (e.g. `awk` on UNIX) — Prune, May 08 '19 at 22:38
(though `awk` uses the `read()` syscall, just like Python and *every other userspace program that interacts with file-like objects on UNIX*, so you won't see a significant improvement in one language's blocking on the call vs another's). — Charles Duffy, May 09 '19 at 01:55
That said, reading from a *device* as opposed to a regular file is going to have performance characteristics that depend on the (typically kernelspace) driver for that device, and the hardware that's behind it. If the hardware sends an update every second and the driver blocks until the next one is available? Guess what, you're waiting up to a second... and it has nothing to do with what programming language you're using. — Charles Duffy, May 09 '19 at 01:57

score 1 · Accepted Answer · answered May 09 '19 at 02:02

You're opening a file that isn't really a regular filesystem file -- it's a character device. Device nodes, on Linux, have their syscalls directly implemented by a specific driver that registers to handle the major/minor number pair, and so their performance is dependent on the operating-system implementation of that driver.

High latency is common to the w1-therm driver; it happens no matter what programming language you use.

Per the data sheet for the hardware, at https://www.maximintegrated.com/en/products/sensors/DS18B20.html, the refresh rate is ~750ms when generating 12-bit outputs. Thus, you're looking at about 3/4 of a second per temperature read, even if everything else is absolutely perfect.

Frankly, a faster refresh rate doesn't make sense in the context of a temperature sensor -- if the physical temperature of the device itself is changing so rapidly that you need more than a measurement every second (taking into account the time needed for the heat to actually be conducted to the sensor), you've got bigger problems.

Why does reading from /sys/bus/... take so much time?

1 Answers1