0

I have a Java program that uploads new/changed files to my Web site via FTP. It currently uses the Apache Commons Net Library, version 3.8.0.

After moving to a new city, the program, which I’ve been using for almost 20 years, began failing. It still connects to the FTP server and signs in successfully. But when it tries to upload a file, it pauses for 20-30 seconds, then fails. It always fails on the first file, 100% of the time.

The failing call is to org.apache.commons.net.ftp.FTPClient.storeFile(). The documentation says storeFile() turns True if successfully completed, false if not. Curiously, the method is also documented to throw various forms of IOException. The documentation doesn’t say when or why the method decides to return a boolean versus throwing an exception.

My problem is that storeFile() is returning a false (indicating failure), and never throws an exception. So, I have no error message to tell me what caused the failure. File name & path look OK. The Web hosting company tried to determine why the failure was occurring, but was unsuccessful.

This problem has been going on for weeks now. Anyone have any ideas on how to debug this?

Olaf Kock
  • 46,930
  • 8
  • 59
  • 90
aksarben
  • 588
  • 3
  • 7

1 Answers1

1

If the cause of your problem is moving to a new city, and you can still open the control connection, the most likely culprit is a change to your underlying ISP and network that is blocking the data transfer stream from opening.

FTP port 21 is used for opening connections and is normally allowed by all networks but then a new, random, unprivileged port is negotiated over the control connection and then used for the actual DATA transfers. I bet your "storeFile()" is trying to open a data connection and hitting a block which is probably causing a timeout. You may be interpretting this as "never throws an exception" but in reality it might throw a Timeout Exception after you sit around and wait long enough.

One thing I would recommend is find a way to have your FTP client use PASSIVE mode for the FTP data transfer. This is designed into the protocol to avoid these types of problems. You can read about it in detail on the wiki https://en.wikipedia.org/wiki/File_Transfer_Protocol under "Communications and Data Transfer"

Atmas
  • 2,389
  • 5
  • 13
  • Well, I know my ISP is not blocking FTP, because I can upload & download files with a different FTP client, such as WinSCP. Will have to dig into my failing client code to see whether it's using PASSIVE mode. – aksarben Mar 19 '21 at 17:44
  • I believe WinSCP uses PASSIVE mode by default,. It shouldn't be terribly hard to set. I looked around and found a sample that should work: Check out https://www.javatips.net/api/org.apache.commons.net.ftp.ftpclient Look for Example 29 - Project: YetAnotherBackupMod-Master File: FTPTask.Java He sets ftp_client.enterLocalPassiveMode(); and then does a storeFile a few lines later. – Atmas Mar 19 '21 at 23:01
  • 1
    Also to clarify, ACTIVE mode dynamically opens ports on your computer to listen for the data connection. This means when you say "my ISP is not blocking FTP", what that means is "The ISP allows port 21 connections in Passive mode to be made from itsi customers to FTP servers for data transfer". This is NOT the same as "The ISP allows customers to open a port 21 connection to a server and then have that server contact the customer back on some random Port 123xyz". Also any NAT/Firewalls on your own PC/Wifi Router would likely block the connection as well. – Atmas Mar 19 '21 at 23:05
  • You were right! Once I dug into the code for the Apache FTPClient class, I discovered its default data connection mode was ACTIVE. After I ensured it was set to PASSIVE before trying to upload a file, everything magically worked (grin). Looks like I was just incredibly lucky that ACTIVE mode worked with my previous ISP. Nota bene: I also discovered that FTPClient resets the mode to ACTIVE during the connect/login process. So I moved my code line that changed the mode to PASSIVE to be right *after* the login, and that's when it started working. Thanks for the insight. – aksarben Mar 20 '21 at 14:48
  • That's awesome! I'm glad it worked out! Please feel free to mark my answer as correct. I would appreciate the kudos! – Atmas Mar 20 '21 at 17:02