What happend if a program read and write to same buffer?

Question

What I encountered is I wrote a sample python script named tt.py:

import time

while True:
    print(123)
    time.sleep(0.5)

I run it by python tt.py &, it will output to terminal 123 on-going. And if I input a character l after several seconds input s, the command ls works fine.

But why, as I know a little bit pseudoterminal: master side and slave side. I thought bash's stdin, stdout, stderr is redirect to slave side. So in my mind, the bash should return command like l123\n123\n...s not found.

So more general question is how bash know which is output which is input when they point to same file descriptor?

Or where I made a misunderstand in this situation, thanks in advance!

Update:

I know stdin and stdout point to different descripor, but they could point to same one right?

So in my mind, python script's stdout to psuedoterminal's slave side, and terminal display streams in master side. Seems what I can't figure out is how terminal stdin goes to bash's stdin. Isn't it first go master side then go slave side then bash?

If so I thougt things will messed up at master side as it receive both python script's output and terminal's input.

@MartijnPieters: fds may be different (standard 0,1) but they may refer to the exact same file: `python3 -c 'import os, sys, pprint; pprint.pprint({(fd:=file.fileno()): os.stat(fd) for file in [sys.stdout, sys.stdin, sys.stderr]})'` — jfs, Nov 28 '20 at 15:28
related: [The TTY demystified](https://www.linusakesson.net/programming/tty/) — jfs, Nov 28 '20 at 15:52
@jfs: sure, but that's not the default; nothing is being redirected here. — Martijn Pieters, Nov 28 '20 at 16:35
@MartijnPieters: it is the default. Try it. It should print inode for your pseudoterminal (assuming you run it in a terminal emulator). https://repl.it/@zed1/standard-streams-file — jfs, Nov 28 '20 at 16:57
@jfs: sorry, yes, misunderstood there. They all point to the same `/dev/pts/...` pseudoterminal device. — Martijn Pieters, Nov 28 '20 at 18:01

Martijn Pieters · Answer 1 · 2020-11-28T21:15:18.993

You are misunderstanding how POSIX standard streams work. There are separate streams for input (stdin), and for normal and error output (stdout and stderr). Those are not the only possible streams.

When you put a process into the background (with &), then that process inherits the same standard streams as bash. So the Python process stdout is the same stream as the bash stdout. Bash is not reading from stdout, however. Your PTY (terminal) is simply displaying all data written to the stdout stream.

To make this explicit: bash is never receiving the data Python writes to stdout, on its stdin. Characters you type in on your keyboard are sent to the bash process stdin stream, entirely separated from the output the Python process is producing.

Note that from the point of view of bash and Python, they are just connected to standard streams (they can test if the stream is connected to a terminal), it doesn't matter if the stream is connected to a real terminal or a pseudo-terminal. The pseudo-terminal is not sending the Python output back to bash either; it is acting like any terminal would; data received from the programs connected is not treated like keyboard input. Both the master and slave sides of the PTY know how to handle input and output and keep them separate:

Generally speaking, you see the ls echoed on your screen because the PTY is set to echo input (bash is not writing this back out after receiving the input).

Yeah, so this question is caused by I misunderstood the stuff I found about pseudo-terminal. Thanks for your explanation. — roachsinai, Nov 28 '20 at 16:39
@roachsinai: no problem; I added a small graph that may help too. — Martijn Pieters, Nov 28 '20 at 17:34
The diagram is helpful. Note: if python process should inherit bash’s stdin by default (for a background job, the process gets suspended if tries to actually read from the terminal) — jfs, Nov 28 '20 at 22:25

Barmar · Accepted Answer · 2020-11-28T16:13:27.197

3

A pseudo terminal consists of two independent channels of data, and they each have their own buffers.

There's one channel that transfers data that's written to the slave pty to the master pty. There's another channel that transfers data written to the master pty to the slave pty.

Anything written to the slave side of the pty is read by the process connected to the master side. In this case, it's your terminal application. It displays this data in the terminal window.

Both python and ls are writing to the slave pty, and their outputs get displayed in the window.

When an application reads from the slave pty, it gets what the other process writes to the master side. The terminal application writes what you type on the keyboard, not the output that was sent to the slave side.

So the output of the python script won't be read as input by the shell.

If the two directions weren't independent, every program that produces output and also reads input would get totally confused, since it would end up reading its own output.

edited Nov 28 '20 at 16:13

answered Nov 28 '20 at 15:27

Barmar

741,623
53
500
612

as you posted: "When an application reads from the slave pty, it gets what the other process writes to the master side. " I though when bash read from the slave side how to make sure the master side is empty except for command `ls`? – roachsinai Nov 28 '20 at 15:51
I don't understand your question. It doesn't need to make sure of anything. It just reads everything you type. First you type `python tt.py&`, it reads that and executes the command. Then you typre `ls`, it reads that, and executes it. Every time you type something, it reads it and executes it. – Barmar Nov 28 '20 at 15:59
A binary like `bash`, `ls` or others, they have stdin, stdout, stderr, it's seperate. What I know is pty master is only a file descriptor so it can't manage stdin and stdout simultaneously (I thought I was wrong here, but didn't know where). But the example I post is the pty master has both stdin and stdout simultaneously. – roachsinai Nov 28 '20 at 16:03
Maybe I got it, the stdout of slave if terminal window? as the 1st paragraph. – roachsinai Nov 28 '20 at 16:10
Just as you can open a file multiple times, you can also open a pty multiple times. – Barmar Nov 28 '20 at 16:10
The pty doesn't "manage stdin". It's just a device driver, it processes data that's written to it, and returns data when a process reads from it. – Barmar Nov 28 '20 at 16:11
1

There are separate buffers in the pty for slave->master and master->slave communications. – Barmar Nov 28 '20 at 16:13
Thanks a lot for your explain! So what you mean for my example is there are several exchange about slave->master(output 123), master->slave(input `l`), slave->master(output 123), master->slave(input `s`)? – roachsinai Nov 28 '20 at 16:32
the fact that bash does not see/try to execute data written but its jobs that it started should not be related to pseudo terminals at all -- it is true however bash is normally run. – jfs Nov 28 '20 at 17:08
@jfs The OP thought that the output of python would become input to bash. – Barmar Nov 28 '20 at 17:15
I know, [e.g., look at the quote at the top of my answer](https://stackoverflow.com/a/65052223/4279). My point is that the output of python would not become input to bash whether pseudoterminals are used or not (under normal circumstances). – jfs Nov 28 '20 at 17:20

score 1 · Answer 3 · answered Nov 28 '20 at 17:00

So in my mind, the bash should return command like l123\n123\n...s not found.

bash in the interactive mode runs commands that are provided on its stdin (e.g., keyboard input). The child process (python in this case) writes to stdout (e.g., display) that it has inherited from bash. stdout data does not go into stdin by default (though it can be done if you'd like).

It doesn't matter in this case whether python is bash's background/foreground job (it doesn't change where python's output goes and we don't care about stty tostop here) or whether a pseudoterminal is used or you're looking at the console directly. bash may use fork, exec syscalls to start child processes & dup2 to configure their stdin/stdout/stderr streams e.g, a pipeline implemented via recursive calls -- no redirection for a single command.

What happend if a program read and write to same buffer?

3 Answers3