1

I am using pysftp to upload a local file that is processed remotely, then returned to the SFTP, where I can download it. At the moment, I cannot get the code to recognize that the processed remote file has been uploaded. The code just waits forever, despite the remotely processed file successfully being uploaded.

While the code is running, if I refresh the connection on a service such as FileZilla, the code instantly recognizes the file is in and works perfectly. However, I need it to work without manually refreshing on FileZilla. I am not sure why the code cannot recognize itself that the file has been uploaded and is ready to be downloaded.

I've already tried disconnecting then re-connecting to the SFTP using a while statement, which was not successful. 

import pysftp
import stat

cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
srv = pysftp.Connection(host="host", username="username", password="passowrd", cnopts=cnopts)

print("Connection Made!")

# Put File In
srv.chdir(r'/path_to_remote_directory')

srv.put(r'C:\Path_to_local_file.csv')
print("File Uploaded")

while not srv.isfile(r'/Path_to_newly_uploaded_remote_file.csv'):
    time.sleep(3)
    print("Still Waiting")

if srv.isfile(r'/Path_to_newly_uploaded_remote_file.csv'):
    print("File is Ready!")
    srv.get('/Path_to_newly_uploaded_remote_file.csv', r'C:\Local_path_save_to/file.csv')

    # Closes the connection
    srv.close()
glibdud
  • 7,550
  • 4
  • 27
  • 37
David Seroy
  • 179
  • 1
  • 14

2 Answers2

1

Just remove the /:

srv.isfile(r'/Path_to_newly_uploaded_remote_file.csv'):
->
srv.isfile(r'Path_to_newly_uploaded_remote_file.csv'):

NOTE:

Do not set cnopts.hostkeys = None, unless you do not care about security. You lose a protection against Man-in-the-middle attacks by doing so.

I've implemented auto_add_key in my pysftp github fork.

auto_add_key will add the key to known_hosts if auto_add_key=True
Once a key is present for a host in known_hosts this key will be checked.

Please refer to Martin Prikryl -> answer about security concerns.

Why using context manager is a good approach:

import pysftp
with pysftp.Connection(host, username="whatever", password="whatever", auto_add_key=True) as sftp:
    #do your stuff here
#connection closed

EDIT: Checked with my code and / was the problem ... please check spelling of your file and that you are in the correct working dir getcwd()_

import pysftp
import stat
import time

cnopts = pysftp.CnOpts()
cnopts.hostkeys = None #do not do this !!!
srv = pysftp.Connection("host", username="user", password="pass", cnopts=cnopts)

print("Connection Made!")

# Put File In
srv.chdir(r'/upload')

srv.put(r'C:\Users\Fabian\Desktop\SFTP\testcsv.csv')
print("File Uploaded")

print(srv.getcwd()) #get the current folder ! <- you should looking here

while not srv.isfile(r'testcsv.csv'):
    time.sleep(3)
    print("Still Waiting")

if srv.isfile(r'testcsv.csv'):
    print("File is Ready!")
    srv.get('testcsv.csv', r'C:\Users\Fabian\Desktop\SFTP\testcsv_new.csv')

    # Closes the connection
    srv.close()
Fabian
  • 1,130
  • 9
  • 25
  • Tried removing the / as recommended and it did not work. It actually prevented the code from working even once I refreshed the SFTP in FileZilla. For clarification, the original code works great when I refresh the directory manually, it's just that it won't recognize the file has been uploaded without doing that. – David Seroy Aug 29 '19 at 20:17
  • @DavidSeroy Had your paths been really as your question shows, then this answer makes sense. If they are not, then you should edit the question to show more relevant path placeholders not to confuse us. – Martin Prikryl Aug 30 '19 at 05:58
0

Found a working solution. I think it was caused by the old directory being cached. By pointing to the new directory every time for an update, it appears to work. Obvious solution now but took me a while. Posting in case anyone else runs into this.

while not srv.isfile(r'/Path_to_newly_uploaded_remote_file.csv'):
    time.sleep(3)
    srv.chdir(r'/Path_to_newly_uploaded_remote_file')
    print("Still Waiting")
David Seroy
  • 179
  • 1
  • 14