How to keep ssh session open after logging in using subprocess.popen?

Question

I am new to Python.

I am trying to SSH to a server to perform some operations. However, before performing the operations, i need to load a profile, which takes 60-90 seconds. After loading the profile, is there a way to keep the SSH session open so that i can perform the operations later?

p = subprocess.Popen("ssh abc@xyz'./profile'", stdout=subprocess.PIPE, shell=True)
result = p.communicate()[0]
print result
return result

This loads the profile and exits. Is there a way to keep the above ssh session open and run some commands?

Example:

 p = subprocess.Popen("ssh abc@xyz'./profile'", stdout=subprocess.PIPE, shell=True)
    <More Python Code>
    <More Python Code>
    <More Python Code>
 <Run some scripts/commands on xyz server non-interactively> 

After loading the profile, I want to run some scripts/commands on the remote server, which I am able to do by simply doing below:

 p = subprocess.Popen("ssh abc@xyz './profile;**<./a.py;etc>**'", stdout=subprocess.PIPE, shell=True)

However, once done, it exists and the next time I want to execute some script on the above server, I need to load the profile again (which takes 60-90 seconds). I am trying to figure out a way where we can create some sort of tunnel (or any other way) where the ssh connection remains open after loading the profile, so that the users don't have to wait 60-90 seconds whenever anything is to be executed.

I don't have access to strip down the profile.

Answer 1

You have to construct a ssh command like this ['ssh', '-T', 'host_user_name@host_address'] then follow below code.

Code:

from subprocess import Popen, PIPE

ssh_conn = ['ssh', '-T', 'host_user_name@host_address']

# if you have to add port then ssh_conn should be as following
# ssh_conn = ['ssh', '-T', 'host_user_name@host_address', '-p', 'port']

commands = """            
    cd Documents/
    ls -l
    cat test.txt
"""

with Popen(ssh_conn, stdin=PIPE, stdout=PIPE, stderr=PIPE, universal_newlines=True) as p:
    output, error = p.communicate(commands)
    print(output)
    print(error)
    print(p.returncode)
    # or can do following things
    p.stdin.write('command_1')
    # add as many command as you want
    p.stdin.write('command_n')

Terminal Output:

Please let me know if you need further explanations.

N.B: You can add command in commands string as many as you want.

Answer 2

Try an ssh library like asyncssh or spur. Keeping the connection object should keep the session open.

You could send a dummy command like date to prevent the timeout as well.

Answer 3

What you want to do is write/read to the process's stdin/stdout.

from subprocess import Popen, PIPE
import shlex

shell_command = "ssh user@address"
proc = Popen(shlex.split(shell_command), stdin=PIPE, universal_newlines=True)

# Do python stuff here

proc.stdin.write("cd Desktop\n")
proc.stdin.write("mkdir Example\n")

# And so on

proc.stdin.write("exit\n")

You must include the trailing newline for each command. If you prefer, print() (as of Python 3.x, where it is a function) takes a keyword argument file, which allows you to forget about that newline (and also gain all the benefits of print()).

print("rm Example", file=proc.stdin)

Additionally, if you need to see the output of your command, you can pass stdout=PIPE and then read via proc.stdout.read() (same for stderr).

You may also want to but the exit command in a try/finally block, to ensure you exit the ssh session gracefully.

Note that a) read is blocking, so if there's no output, it'll block forever and b) it will only return what was available to read from the stdout at that time- so you may need to read repeatedly, sleep for a short time, or poll for additional data. See the fnctl and select stdlib modules for changing blocking -> nonblocking read and polling for events, respectively.

Answer 4

Hello Koshur!

I think that what you are trying to achieve looks like what I've tried in the past when trying to make my terminal accessible from a private website:

I would open a bash instance, keep it open and would listen for commands through a WebSocket connection.

What I did to achieve this was using the O_NONBLOCK flag on STDOUT.

Example

import fcntl
import os
import shlex
import subprocess

current_process = subprocess.Popen(shlex.split("/bin/sh"), stdin=subprocess.PIPE,
                                   stdout=subprocess.PIPE, stderr=subprocess.STDOUT)  # Open a shell prompt
fcntl.fcntl(current_process.stdout.fileno(), fcntl.F_SETFL,
            os.O_NONBLOCK)  # Non blocking stdout and stderr reading

What I would have after this is a loop checking for new output in another thread:

from time import sleep
from threading import Thread

def check_output(process):
    """
    Checks the output of stdout and stderr to send it to the WebSocket client
    """
    while process.poll() is None: # while the process isn't exited
        try:
            output = process.stdout.read() # Read the stdout PIPE (which contains stdout and stderr)
        except Exception:
            output = None
        if output:
            print(output)
        sleep(.1)
    # from here, we are outside the loop: the process exited
    print("Process exited with return code: {code}".format(code=process.returncode))

Thread(target=check_output, args=(current_process,), daemon=True).start() # Start checking for new text in stdout and stderr

So you would need to implement your logic to SSH when starting the process:

current_process = subprocess.Popen(shlex.split("ssh abc@xyz'./profile'"), stdin=subprocess.PIPE,
                                   stdout=subprocess.PIPE, stderr=subprocess.STDOUT)

And send commands like so:

def send_command(process, cmd):
    process.stdin.write(str(cmd + "\n").encode("utf-8")) # Write the input to STDIN
    process.stdin.flush() # Run the command

send_command(current_process, "echo Hello")

EDIT

I tried to see the minimum Python requirements for the given examples and found out that Thread(daemon) might not work on Python 2.7, which you asked in the tags.

If you are sure to exit the Thread before exiting, you can ignore daemon and use Thread() which works on 2.7. (You could for example use atexit and terminate the process)

References

• fcntl(2) man page
  https://man7.org/linux/man-pages/man2/fcntl.2.html
• fcntl Python 3 Documentation
  https://docs.python.org/3/library/fcntl.html
• fcntl Python 2.7 Documentation
  https://docs.python.org/2.7/library/fcntl.html
• O_NONBLOCK Python 3 Documentation
  https://docs.python.org/3/library/os.html#os.O_NONBLOCK
• O_NONBLOCK Python 2.7 Documentation
  https://docs.python.org/2.7/library/os.html#os.O_NONBLOCK