non-blocking stdin using selectors

Question

Playing around with non-blocking console input, using Python's selectors in combination with sys.stdin, there is something I do not understand:

Suppose I want to exit a loop when a user presses Enter, possibly after entering some other characters first.

If I do a blocking read, as follows, the process always finishes after the first linefeed \n it encounters, as expected, regardless of any preceding characters:

import sys

character = ''
while character != '\n':
    character = sys.stdin.read(1)

Now consider the following minimized example of a non-blocking read:

import sys
import selectors

selector = selectors.DefaultSelector()
selector.register(fileobj=sys.stdin, events=selectors.EVENT_READ)

character = ''
while character != '\n':
    for key, __ in selector.select(timeout=0):
        character = key.fileobj.read(1)

If I hit Enter as the first input, that generates a linefeed character, and the process finishes, as expected.

However, if I input some other characters first, followed by Enter, the process does not finish: I need to hit Enter again before it does.

Apparently this implementation only works if the linefeed is the first input.

There's probably a good reason for this, but I do not see it at the moment, and could not find any related questions.

Does this have to do with my non-blocking implementation, or is it a stdin buffer thing, or perhaps something to do with the console or terminal implementation?

(I am running this from a python 3.8 shell on ubuntu.)

Answer 1

sys.stdin is an instance of io.TextIOWrapper which in turn wraps an instance of io.BufferedReader.

Adding print(repr(character)) to the end of your code let's you see what is happening:

$ python foo.py 
# enter asd\n
'a'
# enter \n
's'
'd'
'\n'

When you first enter "asd\n", python read all of it to a buffer, but only returns the first character. Since the underlying stdin is empty now, the next call to selector.select() blocks again. When you enter another newline, select unblocks and the code continues to read characters from the buffer until the first "\n" is reached.

You can bypass the buffering by using os.read(key.fileobj.fileno(), 1). This gives the expected behavior (note that it returns bytes instead of strings):

$ python foo.py 
# enter asd\n
b'a'
b's'
b'd'
b'\n'

Edit: some context about cbreak mode

From man 3 cbreak:

Normally, the tty driver buffers typed characters until a newline or carriage return is typed. The cbreak routine disables line buffering and erase/kill character-processing (interrupt and flow control characters are unaffected), making characters typed by the user immediately available to the program. The nocbreak routine returns the terminal to normal (cooked) mode.

In cbreak mode, the characters are sent one at a time, so python has no chance to buffer the input:

$ python foo.py 
# enter a
'a'
# enter s
's'
# enter d
'd'
# enter \n
'\n'