1

My code uses Paramiko to login to a remote Unix host

  1. grep a string on a remote Linux host then write results to a new file /tmp/file.<timestamp>.txt. – This works fine
  2. sftp to the remote Linux host and get the file I just created from my grep. – This is the step I am having trouble with.

Remote system = Linux Local system = Windows

The problem is when I sftp get the file from my Unix host, the file is empty once it gets to my Windows system.

Can I execute the command which creates the file and do an SFTP put in one go? If so any example on how to do so while putting it in a specific dir on my local Windows system? Or is there something I need to do with the the stdout from the first command? Although I don't think that will work.

butler_report_dir = Path.home() / 'butler_reports'
butler_report_dir.mkdir(exist_ok=True)
req_date = req_date.strftime('%Y%m%d')
req_date = '*' + req_date + '*'
r_cat_file = '/tmp/cat_report_' + timestamp + '.txt'
ssh_client = SSHClient()
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
     
ssh_client.connect(
                   ssh_server,
                   username=ssh_username,
                   password=ssh_passwd,
                   look_for_keys=False,
                  )

cmd_stmt = f'cd {old_arch_dir};' f'/usr/bin/gzip -cd {req_date} | grep {orderid} > {r_cat_file}'
stdin, stdout, stderr = ssh_client.exec_command(cmd_stmt)
# The Code Above seems to work fine as my file is being created with the timestamp
   
# This part below is where I am having the issue
file = butler_report_dir / 'cat_file.txt'
ftp_client = ssh_client.open_sftp()
stdin.close()
stdout.close()
stderr.close()
ssh_client.close()
Martin Prikryl
  • 188,800
  • 56
  • 490
  • 992
Will_Roberts
  • 21
  • 1
  • 6

1 Answers1

1

You are not waiting for the command to finish. So you are downloading an incomplete file.

To wait for the command to finish, you can do the following:

stdin, stdout, stderr = ssh_client.exec_command(cmd_stmt)

stdout.channel.set_combine_stderr(True)
output = stdout.readlines()

ftp_client = ssh_client.open_sftp()

For more, see Wait to finish command executed with Python Paramiko.

Obligatory warning: Do not use AutoAddPolicy – You are losing a protection against MITM attacks by doing so. For a correct solution, see Paramiko "Unknown Server".

Martin Prikryl
  • 188,800
  • 56
  • 490
  • 992