Check for extra characters in Linux terminal buffer

Question

I try to implement getch() function in Python, which should also return list of chars for special keys like F1-F12 and arrow keys. These special keys generate several chars in a sequence. Therefore getch() reads one char in blocking mode and then should check if there are extra chars in input buffer to fetch them too.

I am using ioctl call together with termios.FIONREAD to get the number of bytes in the input buffer. It catches non-special key presses stacked in buffer, but misses extra symbols from special keys. It seems like there are two different buffers and it would be nice if somebody could explain this.

Here is the interactive example:

from time import sleep

def getch():
    import sys, tty, termios
    fd = sys.stdin.fileno()
    # save old terminal settings, because we are changing them
    old_settings = termios.tcgetattr(fd)
    try:
        # set terminal to "raw" mode, in which driver returns
        # one char at a time instead of one line at a time
        #
        # tty.setraw() is just a helper for tcsetattr() call, see
        # http://hg.python.org/cpython/file/c6880edaf6f3/Lib/tty.py
        tty.setraw(fd)
        ch = sys.stdin.read(1)

        # --- check if there are more characters in buffer
        from fcntl import ioctl
        from array import array

        sleep(1)
        buf = array('i', [0])
        ioctl(fd, termios.FIONREAD, buf)
        print "buf queue: %s," % buf[0],
        # ---

    finally:
        # restore terminal settings. Do this when all output is
        # finished - TCSADRAIN flag
        termios.tcsetattr(fd, termios.TCSADRAIN, old_settings)
    return ch

char = ''
while char != 'q':
  char = getch()
  print 'sym: %s, ord(%s)' % (char, ord(char))

Note sleep(1) in the middle. If you hit one key before this second expires, the output will be:

buf queue: 0, sym: l, ord(108)

For 5 ordinary keys (for example 'asdfg') entered in one second, the output is:

buf queue: 4, sym: a, ord(97)

but for a single arrow key, the output:

buf queue: 0, sym: , ord(27)
buf queue: 0, sym: [, ord(91)
buf queue: 0, sym: D, ord(68)

There are two questions here:

1. Why 4 symbols in queue with ordinary key presses are discarded? Is it because of switch to "raw" terminal mode? How is it possible to preserve chars for subsequent getch() runs without leaving terminal in "raw" mode?

2. Why the ioctl buffer for a single special key press is empty? Where are those characters are coming from for subsequent getch() runs? How to check for them?

Answer 1

I ran into this same issue. Some searching yielded a working example that read at most 4 bytes (instead of your 1) to allow for special escape sequences and used os.read (instead of your file.read). Based on those differences, I was able to write a little Keyboard class that recognized cursor key events:

#!/usr/bin/env python

import os
import select
import sys
import termios

class Keyboard:
  ESCAPE = 27
  LEFT = 1000
  RIGHT = 1001
  DOWN = 1002
  UP = 1003

  keylist = {
    '\x1b' : ESCAPE,
    '\x1b[A' : UP,
    '\x1b[B' : DOWN,
    '\x1b[C' : RIGHT,
    '\x1b[D' : LEFT,
  }

  def __init__(self):
    self.fd = sys.stdin.fileno()
    self.old = termios.tcgetattr(self.fd)
    self.new = termios.tcgetattr(self.fd)
    self.new[3] = self.new[3] & ~termios.ICANON & ~termios.ECHO
    self.new[6][termios.VMIN] = 1
    self.new[6][termios.VTIME] = 0
    termios.tcsetattr(self.fd, termios.TCSANOW, self.new)

  def __enter__(self):
    return self

  def __exit__(self, type, value, traceback):
    termios.tcsetattr(self.fd, termios.TCSAFLUSH, self.old)

  def getFile(self):
    return self.fd

  def read(self):
    keys = os.read(self.fd, 4)
    if keys in Keyboard.keylist:
      return Keyboard.keylist[keys]
    else:
      return None

if __name__ == "__main__":
  with Keyboard() as keyboard:
    key = keyboard.read()
    while key != Keyboard.ESCAPE:
      print '%d' % key
      key = keyboard.read()

With file.read(4), the reading blocks. With os.read(fd, 4), the reading does not block. I don't know why there's a difference and would welcome enlightenment.