2

The essence of my question is "How can a user program open a pty (e.g., /dev/ptyp0) on Mac OS High Sierra?" A standard open() of the device name does not seem to work, even though its protections are crw-rw-rw-.

The context is running Emacs under Mac OS. Before High Sierra, when I ordered Emacs to create a shell window within which is a subprocess, Emacs allocated a pty with which to communicate with the subprocess. After upgrading to High Sierra, Emacs can no longer allocate the pty, and uses pipes to communicate with the subprocess. This makes the subprocess act less like a shell-in-a-terminal-window, which is a poorer user experience.

Tracing Emacs' operations shows that Emacs is trying in sequence to open each of the ptys defined on the system, and receiving errno 35 each time.

I have located one item online that suggests that how one opens pty's on Mac OS High Sierra is different from previous versions of Mac OS, but it gave no details.

user2864482
  • 221
  • 1
  • 4

1 Answers1

1

The essence of the answer is that there are at least two mechanisms used on Unix-like systems to allocate ptys. High Sierra uses a different mechanism than previous Mac OS versions, one involving the device /dev/ptmx apparently, and so if your program does not use the correct mechanism, it will fail. In Emacs' case, if it fails to allocate a pty, it falls back to using pipes to communicate with a subprocess, which subprocess then does not "have a tty" and you cannot use tty functions on it.

In regard to Emacs, I was using version 22.1.1, which was configured to use the previous pty mechanism (attempting to open each /dev/ptyXX in sequence). The problem is fixed by using version 26.1. You can download version 26.1 of Emacs built as a Mac application from https://emacsformacosx.com

user2864482
  • 221
  • 1
  • 4