How to execute a command and read/write to its STDIN/TTY (together)?

Question

I've seen examples and questions about how to do these things individually. But in this question I'm trying to do them all jointly.

Basically my case is that I have a command that needs me to write to its STDIN, read from its STDOUT, and to answer its TTY prompts. All done with a single execution of the command. Not that it matters, but if you're curious, the command is scrypt enc - out.enc.

Restrictions: must be pure Python.

Question: how to do it?

I tried these:

import pty
import os
import subprocess

master, slave = pty.openpty()
p = subprocess.Popen(['sudo', 'ls', '-lh'], stdin=slave, stdout=master)

x= os.read(master)
print(x)

stdout, stderr = p.communicate(b'lol\r\n')

import pty
import os
import sys
import subprocess

def read(fd):
    data = os.read(fd, 1024)
    data_str = data.decode()
    if data_str.find('[sudo] password for') == 0:
        data_str = 'password plz: '
    sys.stdout.write(data_str)
    sys.stdout.flush()

def write(fd):
    x = 'lol\r\n'
    for b in x.encode():
        os.write(fd, b)

pty.spawn(['sudo', 'ls', '-lh'], read, write)

The goal is to fully wrap the TTY prompts so that they are not visible to the user, and at the same time to feed some password to processes TTY input to make sudo happy.

Based on that goal, none of these attempts work for various reasons.

But it is even worse: suppose that they work, how can I feed the process something to its STDIN and its TTY-input? What confuses me is that the Popen example literally states that stdin is mapped to TTY (pty), so how can it know which is which? How will it know that some input is for STDIN and not TTY-in?

Please add some information what you have tried so far. – toydarian Sep 03 '20 at 11:07 — toydarian, Sep 03 '20 at 11:07

score 2 · Answer 1 · answered Oct 02 '20 at 10:11

Disclaimer:
Discussing this topic in detail would require a lot of text so I will try to simplify things to keep it short. I will try to include as many "for further reading" links as possible.

To make it short, there is only one input stream, that is STDIN. In a normal terminal, STDIN is connected to a TTY. So what you "type on TTY" will be read by the shell. The shell decides what to do with it then. It there is a program running, it will send it to STDIN of that program.
If you run something with Popen in python, that will not have a tty. You can check that easily by doing this:

from subprocess import Popen, PIPE
p = Popen("tty", stdin=PIPE, stdout=PIPE, stderr=PIPE)
o, e = p.communicate()
print(o)

It will produce this output: b'not a tty\n'

But how does scrypt then try to use a TTY? Because that is what it does.
You have to look at the manpage and code, to find the answer.

If -P is not given, scrypt reads passphrases from its controlling terminal, or failing that, from stdin.

What it does is actually, it is just opening /dev/tty (look at the code). That exists, even if the process does not have a TTY. So it can open it and it will try to read the password from it.

How can you solve your problem now?
Well, that is easy in this case. Check the manpage for the -P parameter.
Here is a working example:

from subprocess import Popen, PIPE
p = Popen("scrypt enc -P - out.enc", stdin=PIPE, stdout=PIPE, stderr=PIPE, universal_newlines=True, shell=True)
p.communicate("pwd\nteststring")

This will encrypt the string "teststring" with the password "pwd".

There are a lot of "hacks" around ttys etc. but you should avoid those as they can have unexpected results. For example, start a shell and run tty then run a second shell and run cat with the output of the tty command (e.g. cat /dev/pts/7). Then type something in the first shell and watch what happens.
If you don't want to try it out, some characters will end up in the first shell, some in the second.

Check this post and this article about what a TTY is and where it comes from.

Did you try your example? Because `-P` cannot be used with `-` with `scrypt`. The combination causes the error `scrypt: Cannot read both passphrase and input file from standard input`. — caveman, Oct 19 '20 at 18:34
@caveman I have the following scrypt version: `scrypt 1.2.0-head` — toydarian, Oct 20 '20 at 08:13
interesting. `scrypt 1.3.1` here. I'm now thinking to replace `-` by a named pipe in a temporary directory, and hope that no other application reads it before `scrypt`. Alternatively, use `argon2id` to derive a key from a password (works with STDIN), then use that derived key with whatever nice sharedkey cipher in Python. — caveman, Oct 20 '20 at 11:13
@caveman why not read the input in python until receiving EOF, write it to a temporary file and encrypt that? — toydarian, Oct 20 '20 at 11:16
How to hand `scrypt` the password? My application has also the feature of updating the file, so I chose to prompt the password from the user and store it in a variable, then every time the app is run, the password is prompted once, then my script feeds the stored password to `scrypt` for as much is needed. If I don't do this, then the user will have to type the password to decrypt the database, and encrypt it again (typing password twice adds a user error. — caveman, Oct 20 '20 at 11:32
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/223353/discussion-between-toydarian-and-caveman). — toydarian, Oct 20 '20 at 11:50

How to execute a command and read/write to its STDIN/TTY (together)?

1 Answers1