0

I have been having trouble with a File upload via FTP. I can upload small files and get a 226 stored message that everything works. When I run a larger file (200mbs) that takes about 15 minutes to upload, I see that the file uploads when I look at FileZilla but I get the following error that breaks my try/except in python:

[WinError 10054] An existing connection was forcibly closed by the remote host

Here is my script (Thanks to this answered question, I was able to actually get my files uploaded using SmartFTP - Cannot list FTP directory using ftplib – but FTP client works):

class SmartFTP(FTP):
    def makepasv(self):
        invalidhost, port = super(SmartFTP, self).makepasv()
        return self.host, port


try:
    for file_name in os.listdir(fullPath):
        if file_name == 'file.csv':
            
            file_path = fullPath + "\\" + file_name
            print('Moving File - ' + file_path + ': ' + str(datetime.now()))
            
            with SmartFTP(FTP_HOST, FTP_USER, FTP_PASS, timeout=2700) as ftp, open(file_path, 'rb') as file:
                ftp.set_debuglevel(2)
                print('here1')
                test = ftp.pwd()
                print('here2')
                ftp.storbinary(f'STOR {file_name}', file)
                print('here3')
                print('FTP Move Completed')
                

except Exception as e:
    print("FTP and File Move process failed: " + str(e))

Moving File - file.csv: 2022-07-12 15:27:35.932522

here1

*cmd* 'PWD'

*put* 'PWD\r\n'

*get* '257 "/folder"\n'
*resp* '257 "/folder"'
here2
*cmd* 'TYPE I'
*put* 'TYPE I\r\n'
*get* '200 Transfer Mode: BINARY\n'
*resp* '200 Transfer Mode: BINARY'
*cmd* 'PASV'
*put* 'PASV\r\n'
*get* '227 Entering PASV Mode (numbers)\n'
*resp* '227 Entering PASV Mode (numbers)'
*cmd* 'STOR file.csv'
*put* 'STOR file.csv\r\n'
*get* '150 Connecting Data Port...\n'
*resp* '150 Connecting Data Port...'

Then it loads for about 15 minutes

*cmd* 'QUIT'
*put* 'QUIT\r\n'
FTP and File Move process failed: [WinError 10054] An existing connection was forcibly closed by the remote host

This what happens when I load a smaller file:

Moving File - file.csv: 2022-07-12 15:50:39.489980
here1
*cmd* 'PWD'
*put* 'PWD\r\n'
*get* '257 "/Atlas_In"\n'
*resp* '257 "/Atlas_In"'
here2
*cmd* 'TYPE I'
*put* 'TYPE I\r\n'
*get* '200 Transfer Mode: BINARY\n'
*resp* '200 Transfer Mode: BINARY'
*cmd* 'PASV'
*put* 'PASV\r\n'
*get* '227 Entering PASV Mode (numbers)\n'
*resp* '227 Entering PASV Mode (numbers)'
*cmd* 'STOR file.csv'
*put* 'STOR file.csv\r\n'
*get* '150 Connecting Data Port...\n'
*resp* '150 Connecting Data Port...'
*get* '226 Stored Message: [file.csv] Byte Count: [57824] bytes\n'
*resp* '226 Stored Message: [file.csv] Byte Count: [57824] bytes'
here3
FTP Move Completed
*cmd* 'QUIT'
*put* 'QUIT\r\n'
*get* '221 GoodBye\n'
*resp* '221 GoodBye'

They have a process that moves files from the folder I'm posting, into a downloaded folder. Is this what is causing the issue?

I have tried setting the ftp.set_pasv(False), but this equates in the process loading forever (no end).

As requested, here is the log (and error) of what happens when loaded via FileZilla:

Status: Resolving address of web.com
Status: Connecting to IP:21...
Status: Connection established, waiting for welcome message...
Status: Insecure server, it does not support FTP over TLS.
Status: Logged in
Status: Retrieving directory listing...
Status: Directory listing of "/directory" successful
Status: Resolving address of web.com
Status: Connecting to IP:21...
Status: Connection established, waiting for welcome message...
Status: Insecure server, it does not support FTP over TLS.
Status: Logged in
Status: Starting upload of file.csv
Status: Server sent passive reply with unroutable address. Using server address instead.
Status: Disconnected from server: ECONNABORTED - Connection aborted
Command:    STOR file.csv
Response:   150 Connecting Data Port...
Error:  Disconnected from server: ECONNABORTED - Connection aborted
Error:  File transfer failed after transferring 90,439,680 bytes in 303 seconds

Fails here then retries (filezilla settings are to try twice I believe)

Status: Resolving address of web.com
Status: Connecting to IP:21...
Status: Connection established, waiting for welcome message...
Status: Insecure server, it does not support FTP over TLS.
Status: Logged in
Status: Starting upload of file.csv
Status: Retrieving directory listing of "/directory"...
Status: Server sent passive reply with unroutable address. Using server address instead.
Status: Server sent passive reply with unroutable address. Using server address instead.

Is there anything else I can try? Thanks so much for any help!

  • 1
    Can you upload the larger file using any commandline/GUI FTP client running on the same machine as your Python code? Post its verbose log file. – Martin Prikryl Jul 14 '22 at 06:19
  • Thanks @MartinPrikryl added the logging of FileZilla above. It fails after around 300 seconds – Elliot Gitter Jul 14 '22 at 17:48
  • 1
    Well, if FileZilla does not work either, you do not have a [programming question](https://stackoverflow.com/help/on-topic). Please move it to [su] or [sf]. – Martin Prikryl Jul 14 '22 at 19:27
  • Thanks, will check it out @MartinPrikryl. Is there a reason the ftplib isn't asking for the GET command (226) on the large file? Could it be the other server timed out during upload so it's no doing that? That seems to be the catch.. Will let you know if I find out what's going on. – Elliot Gitter Jul 15 '22 at 19:25
  • The `226` is a *response* from the *server*. It does not come, because the connection is aborted. You will probably see the same in FileZilla log file (in the real log file, not in the GUI message log you have posted). – Martin Prikryl Jul 15 '22 at 20:02

0 Answers0