Python read named PIPE

Question

I have a named pipe in linux and i want to read it from python. The problem is that the python process 'consumes' one core (100%) continuously. My code is the following:

FIFO = '/var/run/mypipe'
os.mkfifo(FIFO)
with open(FIFO) as fifo:
    while True:
        line = fifo.read()

I want to ask if the 'sleep' will help the situation or the process going to loss some input data from pipe. I can't control the input so i don't know the frequency of data input. I read about select and poll but i didn't find any example for my problem. Finally, i want to ask if the 100% usage will have any impact on the data input(loss or something?).

edit: I don't want to break the loop. I want the process runs continuously and 'hears' for data from the pipe.

Answer 1

In typical UNIX fashion, read(2) returns 0 bytes to indicate end-of-file which can mean:

• There are no more bytes in a file
• The other end of a socket has shutdown the connection
• The writer has closed a pipe

In your case, fifo.read() is returning an empty string, because the writer has closed its file descriptor.

You should detect that case and break out of your loop:

reader.py:

import os
import errno

FIFO = 'mypipe'

try:
    os.mkfifo(FIFO)
except OSError as oe: 
    if oe.errno != errno.EEXIST:
        raise

print("Opening FIFO...")
with open(FIFO) as fifo:
    print("FIFO opened")
    while True:
        data = fifo.read()
        if len(data) == 0:
            print("Writer closed")
            break
        print('Read: "{0}"'.format(data))

Example session

Terminal 1:

$ python reader.py 
Opening FIFO...
<blocks>

Terminal 2:

$ echo -n 'hello' > mypipe 

Terminal 1:

FIFO opened
Read: "hello"
Writer closed
$ 

---

Update 1 - Continuously re-open

You indicate that you want to keep listening for writes on the pipe, presumably even after a writer has closed.

To do this efficiently, you can (and should) take advantage of the fact that

Normally, opening the FIFO blocks until the other end is opened also.

Here, I add another loop around open and the read loop. This way, once the pipe is closed, the code will attempt to re-open it, which will block until another writer opens the pipe:

import os
import errno

FIFO = 'mypipe'

try:
    os.mkfifo(FIFO)
except OSError as oe:
    if oe.errno != errno.EEXIST:
        raise

while True:
    print("Opening FIFO...")
    with open(FIFO) as fifo:
        print("FIFO opened")
        while True:
            data = fifo.read()
            if len(data) == 0:
                print("Writer closed")
                break
            print('Read: "{0}"'.format(data))

---

Terminal 1:

$ python reader.py 
Opening FIFO...
<blocks>

Terminal 2:

$ echo -n 'hello' > mypipe 

Terminal 1:

FIFO opened
Read: "hello"
Writer closed
Opening FIFO...
<blocks>

Terminal 2:

$ echo -n 'hello' > mypipe 

Terminal 1:

FIFO opened
Read: "hello"
Writer closed
Opening FIFO...
<blocks>

... and so on.

---

You can learn more by reading the man page for pipes:

• PIPE(7) - Linux Programmer's Manual
• FIFO(7) - Linux Programmer's Manual

Answer 2

(Years later) If I'm understanding the OP's use case using for ... in ... does exactly what is desired:

import os

FIFO = 'myfifo'
os.mkfifo(FIFO)
with open(FIFO) as fifo:
    for line in fifo:
        print(line)

This program patiently waits for input from the fifo until is is provided, then prints it on the screen. No CPU is used in the meantime.

This is also the more idiomatic way in Python so I would recommend it rather than using read() directly.

If the client side writing to the fifo closes, the for loop ends and the program quits. If you wanted it to reopen the fifo to wait for the next client to open it you can put the for section into a while loop:

import os

FIFO = 'myfifo'
os.mkfifo(FIFO)
while True:
    with open(FIFO) as fifo:
        for line in fifo:
            print(line)

This will reopen the fifo and wait as usual.