11

I wanted to ask if anyone knows about any Java 7 issues with FTP? I've used both the Sun Net and Apache Commons Net libraries and both perform as expected on Java 6. But when I switch my dev environment (Eclipse) to 1.7, the same operations perform really slow (about 4.5 to 8KB/s), and these are to localhost servers and another server within the LAN.

I've tried buffered streams, byte-to-byte transfer, turning the Nagle Algorithm off, and using the Apache convenience method storeFile(), with the latter finally performing to speed on localhost but slowing down again to a crawl on a remote server. I also set all machines to turn off stateful FTP filtering.

    InputStream is = null;
    OutputStream os = null;
    try {
        is = new BufferedInputStream(prepareInputStream(data));
        os = new BufferedOutputStream(prepareOutputStream(data));
        if (is == null || os == null) {
            log.error("Can't build connection");
            return;
        }

        byte[] buf = new byte[4096];
        int c = 1;

        while (c > 0) {
            c = is.read(buf);
            if (c > 0)
            os.write(buf, 0, c);
            data.incrCurrentPosition();
            fireStateChanged(data);
        }
        data.incrCurrentPosition();
    } catch (IOException e) {
        log.error(e.getMessage(), e);
        setEnabled(false);  
    } catch (Exception e) {
        log.error(e.getMessage(), e);
    } finally {
        if (is != null) {
            try {
                is.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        if (os != null) {
            try {
                os.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }

As can be seen, this is pretty standard implementation code. Again, in Java 6, things zip by really quick. In Java 7, it slows down by a factor of 10 to 20 for both the Sun and Apache Commons libraries. Using an FTP client like FileZilla confirms that FTP is functioning normally, so I think it really has something to do with Java 7. I dug as far as I could online for any mention of a problem but, mostly, the things I saw were about the Java 7 and Windows 7 firewall conflict.

Thanks in advance for any insight given.

Adam Law
  • 542
  • 1
  • 7
  • 21
  • 1
    Q: "I think it really has something to do with Java 7". A: I really think you're probably right. Try compiling for Java 6, and try running the *same* .class on both JRE6 and JRE7. I'll bet you probably confirm your theory :) – paulsm4 Dec 22 '12 at 05:36
  • What's the class of `data`? – DWright Dec 22 '12 at 05:37
  • sorry I missed this DWright. I've tried arbitrary data types from hash mapped objects to strings, just to see if it was indeed the cause but they all transferred the same, that is, fast in Java 6 and slow in Java 7. – Adam Law Dec 22 '12 at 06:01
  • Have you tried profiling with VisualVM to pinpoint the actual method? – Thorn G Dec 22 '12 at 06:39
  • I was setting it up but I also thought to jump in here to see if it's a known issue. – Adam Law Dec 22 '12 at 07:16
  • Well, the active method running through all this is sun.rmi.transport.tcp.TCPTransport%ConnectionHandler.run(), with the most active thread being (unsurprisingly given the slow rate) being the one for RMI TCP accept. While it was good to know, does it then imply something wrong with the Java 7 implementation? – Adam Law Dec 22 '12 at 07:51
  • Testing for null after creating the streams is pointless. They won't be null. Constructing them might throw an exception. Not the same thing. – user207421 Dec 23 '12 at 00:09
  • it's not my code really, I'm doing updates to legacy code and cleaning up stuff like you pointed out is something I'm doing as I go. thanks for pointing it out though. – Adam Law Dec 23 '12 at 06:43
  • It's not great code: don't hesitate. There are two successive calls to data.incrCurrentPosition() at EOS too; the second one is redundant, and probably does nothing in the absence of a following fireStateChanged() call. The buffer size should be increased from 4096. – user207421 Dec 23 '12 at 08:45

3 Answers3

13

I found a fix of sorts, at least enough to get things running normally in Java 7. I did it by using FTPClient's setBufferSize(0); Unfortunately, I don't think there's a similar method in Sun's Java 7's Sun Net implementation. Not that it matters to me as I'm quite pleased with Apache Commons Net. Hopefully, Oracle will get to the bottom of this in due time.

Adam Law
  • 542
  • 1
  • 7
  • 21
  • Thanks, this solved the problem for me. Any idea why this is the case? – Jamie McCrindle Mar 12 '13 at 11:07
  • WOW! I had the same problem and adding ftpClient.setBufferSize(0) dropped upload time for one file from 117s to 6s! That's what I call a good day at the office! – Volksman Apr 01 '13 at 09:21
11

Please check what your current buffer size is with :

ftpClient.getBufferSize();

If you haven't already set it to something else, that will be zero (0). So, set it to a higher value : 

ftpClient.setBufferSize(1048576);//1024*1024

You can check its current value as before :

ftpClient.getBufferSize();

By the way, the accepted answer, setBufferSize(0), did not work for me. I use the latest version of Apache commons, so probably that solution worked with earlier versions. If you set buffer size to zero, there will be no change with the current version.

M-D
  • 10,247
  • 9
  • 32
  • 35
  • 1
    Thanks. I was using 3.2 at the time, and you're right, the issue is fixed in 3.3. Changing the accepted answer to yours. – Adam Law Dec 25 '14 at 23:11
  • I was having some performance problems with bigfile dowloads using camel-ftp, and this pointed me to the right direction. Thanks – rwijngaa Jun 02 '15 at 13:54
5

There is a known performance problem with Commons Net 3.2 which has been fixed in the 3.3 snapshot which you can get from here:

https://repository.apache.org/content/groups/snapshots/commons-net/commons-net/3.3-SNAPSHOT/

While the setBufferSize(0) seems like a valid workaround it's probably best to get the proper fix using the snapshot - if you're ok with using snapshots ;)

Volksman
  • 1,969
  • 23
  • 18