Paramiko .recv is not reading complete output of a command executed with .send

Question

I have a problem where the following function

remote.send("show run int vlan 1\\n")

does not work. The code is below. It is able to read the file, create a new file and write and save the file. However the output only contain the switch name, nothing else and sometimes it will show the switch name and a character or 2 of the command. The output is as follows

switch test1
IP address 1.1.1.51
test1_2023-02-23
Successfully log in to test1
test1#s
switch test2
IP address 1.1.1.50
Test2_2023-02-23
Successfully log in to test2
test2#

Just started to learn the language and I am not sure what the problem is. Thanking you in advance.

myfile = open('c:\znet\device-list.txt','r')

count = 0

while True:
    count = count + 1
    line = myfile.readline()
    if not line:break
    else:
        x = line.strip()
        y,z =x.split(",",1)
        print ("switch ",y)
        print ("IP address ",z)
        backupdate = str(date.today())
        filename = y + "_" + backupdate
        print (filename)
        host = z
        user = 'cisco'
        password = 'cisco'
        client = paramiko.SSHClient()
        client.set_missing_host_key_policy(paramiko.AutoAddPolicy)
        client.connect(hostname=host, username=user, password=password)
        print ("Successfully log in to", y)
        remote = client.invoke_shell()
        remote.send("show run int vlan 1\n")
        output = remote.recv(1048576)
        print (output.decode('ascii'))
        with open("{}.txt".format(filename),"w") as f:
            print (output.decode('ascii'),file=f)

I am expecting to see the configuration of vlan 1 (show run int vlan 1) both on the terminal and in the file created.

Answer 1

The recv won't return you whole output. It cannot. You are using a shell (invoke_shell). The shell is a never ending stream of intermittent data. The only end is the logout. Neither there's any signal that complete output of specific command has ended. The recv returns whatever data are available at the moment you call it. If you want whole output, you need to call recv repeatedly, until you get it all.

This is one of the reasons, why in general, you should not use invoke_shell for command automation.
See What is the difference between exec_command and send with invoke_shell() on Paramiko?

Answer 2

Martin is correct, I've put together some code that will help check and gather data as it's send back by the device to the SSH channel. Code is commented and it should make sense.

import paramiko, time

HOST = "x.x.x.x"
PORT = 22
USERNAME = "xxx"
PASSWORD = "xxx"

def ClearBuffer(connection: paramiko.Channel, sleeptime: float, lookfor: str):
    # This will get rid of all the login information, like banners etc.
    buffer: str = ""
    timer: float = 0.0
    error: int = 0
    errorDescription: str = ""
    while not buffer.endswith(lookfor):
        if timer > 10.0:
            error: int = 1
            errorDescription: str = "Reached command timeout"
            break
        if connection.recv_ready():
            buffer += connection.recv(65535).decode()
        time.sleep(sleeptime)
        timer += sleeptime
    return(buffer) if error != 1 else errorDescription

def ExecuteSingleCommand(connection: paramiko.Channel, command: str, sleeptime: float, lookfor: str):
    buffer: str = ""
    timer: float = 0.0
    error: int = 0
    errorDescription: str = ""
    connection.send(f"{command}\n") # Send command to device and press enter (\n)
    while not buffer.endswith(lookfor): # Continuously look for lookfor, which in this case is #
        if timer > 30.0: # Set a general timer in order to stop execution after 30 seconds
            error = 1
            errorDescription = "Reached command timeout"
            break
        if connection.recv_ready(): # Check if channel has data
            buffer += connection.recv(65535).decode() # Save channel data
            if "--More--" in buffer: # Specific for Cisco devices, if 'terminal length 0' is not entered, channel waits for key press
                errorDescription = "Paging not disabled"
                break
        time.sleep(sleeptime) # wait for sleep time seconds
        timer += sleeptime
    return(buffer.strip(command).strip(lookfor).strip()) if error != 1 else errorDescription

def ExecuteCommand(cmdlist: list, sleeptime: float=0.1, lookfor: str="#"):
    # sleeptime = time to wait before checking for data
    # lookfor = the character to look for in the SSH shell to indicate the command was executed
    results: list = []
    error: int = 0
    with paramiko.SSHClient() as ssh: # Connect to devices using a context manager, which will automatically close connections
        ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
        ssh.connect(HOST, PORT, USERNAME, PASSWORD, timeout=300, banner_timeout=240, look_for_keys=False) # Connect to device
        connection: paramiko.Channel = ssh.invoke_shell() # Invoke shell for interactive commands
        invoke = ClearBuffer(connection, sleeptime, lookfor) # Clear the current channel buffer for login information
        if "Reached command timeout" not in invoke: # Check that clear buffer did not timeout, e.g. it couldn't find lookfor in the buffer
            for cmd in cmdlist: # Iterate every command
                cmdResult = ExecuteSingleCommand(connection, cmd, sleeptime, lookfor) # Execute every single command
                if "Reached command timeout" not in cmdResult: # Check that connection didn't time out
                    results.append(cmdResult) # Append results
                else:
                    error = 1
        else:
            error = 1
    return(results) if error != 1 else None

print(ExecuteCommand(["terminal length 0", "sh run"]))

Test result:

['xxx-xx-3560CG-01', 'Building configuration...\r\n\r\nCurrent configuration : 5893 bytes\r\n!\r\n! Last configuration change at 09:30:59 CET Mon Jan 17 2022 by...TRUNCATED]

You should be able to optimize on the code and easily add additional features yourself.