1

I am running Python 2.7 on a Unix environment (tested on Ubuntu and OSX)

I have the following programs:

With os.open():

[SCRIPT 1]

import os

pipe_1_name = "pipe_1"
pipe_2_name = "pipe_2"
pipe_3_name = "pipe_3"


def set_connection():
    pipe_names = [pipe_1_name, pipe_2_name, pipe_3_name]
    for pipe_name in pipe_names:
        if os.path.exists(pipe_name):
            os.remove(pipe_name)
            os.mkfifo(pipe_name)
        else:
            os.mkfifo(pipe_name)

    pipe_1 = os.open(pipe_1_name, os.O_WRONLY)
    os.write(pipe_1, "server_message_0\n")

    pipe_2 = open(pipe_2_name, 'r')
    received = pipe_2.readline()[:-1]
    print "[0] Now processing if received is correct: " + received

    pipe_3 = open(pipe_3_name, 'r')
    received = pipe_3.readline()[:-1]
    print "[1] Now processing if received is correct: " + received

    print "Connection established."
    return pipe_1,pipe_2,pipe_3


def main():
    pipe_1, pipe_2, pipe_3 = set_connection()

    print str(pipe_1)
    print str(pipe_2)
    print str(pipe_3)

if __name__ == "__main__":
    main()

[SCRIPT 2]

import os

pipe_1_name = "pipe_1"
pipe_2_name = "pipe_2"
pipe_3_name = "pipe_3"


def get_connection():
    pipe_names = [pipe_1_name, pipe_2_name, pipe_3_name]
    for pipe_name in pipe_names:
        if not os.path.exists(pipe_name):
            raise Exception("Pipe "+pipe_name+" does not exist!")

    pipe_1 = open(pipe_1_name, 'r')
    received = pipe_1.readline()[:-1]
    print "[0] Now processing if received is correct: " + received

    pipe_2 = os.open(pipe_2_name, os.O_WRONLY)
    os.write(pipe_2, "client_message_0\n")

    pipe_3 = os.open(pipe_3_name, os.O_WRONLY)
    os.write(pipe_3, "client_message_1\n")

    print "Connection established."
    return pipe_1,pipe_2,pipe_3


def main():
    pipe_1, pipe_2, pipe_3 = get_connection()

    print str(pipe_1)
    print str(pipe_2)
    print str(pipe_3)

if __name__ == "__main__":
    main()

The logic is simple:

[Pipe 1] 1. Script 1 opens a write pipe to Script 2. 2. Script 2 reads from the pipe. [Pipe 2] 3. Script 2 open a write pipe to Script 1. 4. Script 1 reads from the pipe. [Pipe 3] 5. Script 2 open a write pipe to Script 1. 6. Script 1 reads from the pipe.

Works exactly as expected.

Here is the problem. I don't want to use os.open(). I would like the receive a file object and use it to interface with the pipe. Clearly, it is not impossible since I can read from a pipe with a file object. However, The following script does not work.

Without os.open()

[Script 1]

import os

pipe_1_name = "pipe_1"
pipe_2_name = "pipe_2"
pipe_3_name = "pipe_3"


def set_connection():
    pipe_names = [pipe_1_name, pipe_2_name, pipe_3_name]
    for pipe_name in pipe_names:
        if os.path.exists(pipe_name):
            os.remove(pipe_name)
            os.mkfifo(pipe_name)
        else:
            os.mkfifo(pipe_name)

    pipe_1 = open(pipe_1_name, 'w')
    pipe_1.write("server_message_0\n")

    pipe_2 = open(pipe_2_name, 'r')
    received = pipe_2.readline()[:-1]
    print "[0] Now processing if received is correct: " + received

    pipe_3 = open(pipe_3_name, 'r')
    received = pipe_3.readline()[:-1]
    print "[1] Now processing if received is correct: " + received

    print "Connection established."
    return pipe_1,pipe_2,pipe_3


def main():
    pipe_1, pipe_2, pipe_3 = set_connection()

    print str(pipe_1)
    print str(pipe_2)
    print str(pipe_3)

if __name__ == "__main__":
    main()

[Script 2]

import os

pipe_1_name = "pipe_1"
pipe_2_name = "pipe_2"
pipe_3_name = "pipe_3"


def get_connection():
    pipe_names = [pipe_1_name, pipe_2_name, pipe_3_name]
    for pipe_name in pipe_names:
        if not os.path.exists(pipe_name):
            raise Exception("Pipe "+pipe_name+" does not exist!")

    pipe_1 = open(pipe_1_name, 'r')
    received = pipe_1.readline()[:-1]
    print "[0] Now processing if received is correct: " + received

    pipe_2 = open(pipe_2_name, 'w')
    pipe_2.write("client_message_0\n")

    pipe_3 = open(pipe_3_name, 'w')
    pipe_3.write("client_message_1\n")

    print "Connection established."
    return pipe_1,pipe_2,pipe_3


def main():
    pipe_1, pipe_2, pipe_3 = get_connection()

    print str(pipe_1)
    print str(pipe_2)
    print str(pipe_3)

if __name__ == "__main__":
    main()

They look the same, don't they? The only difference is how I open the fifo. Instead of os.open(pipe_name,os.O_WRONLY) I use pipe = open(pipe_name, 'w').

What happens in the second set of scripts, the ones that don't use os.open(), Script 1 blocks at pipe_2 = open(pipe_2_name, 'r') while Script 2 blocks at pipe_2 = open(pipe_2_name, 'w').

Why is this happening?

Sorry for the wall of text. I am really confused about this issue.

RandomGuyqwert
  • 425
  • 6
  • 18

1 Answers1

0

What happens in the second set of scripts, the ones that don't use os.open(), Script 1 blocks at pipe_2 = open(pipe_2_name, 'r') while Script 2 blocks at pipe_2 = open(pipe_2_name, 'w').

No, Script 2 blocks at received = pipe_1.readline()[:-1].

Why is this happening?

It's because Script 1's open(pipe_1_name, 'w') causes the written message to be buffered in fixed-size chunks (typically 4096 or 8192 bytes), so the pipe_1.write("server_message_0\n") does not yet write anything to the pipe, but only to the buffer, and Script 2 doesn't get anything to read. See open() and also How often does python flush to a file?

To cure this, since your messages are complete lines, it suffices to use line buffering, e. g.

    pipe_1 = open(pipe_1_name, 'w', 1)

(as well for the other write pipes).

Armali
  • 18,255
  • 14
  • 57
  • 171