Thread is not disconnecting from server even after its file is downloaded from ftp server and not reusing itself to download another one

Question

This is a code snippet.

FtpDownloader.java

ExecutorService pool = Executors.newFixedThreadPool(5);

 for (FTPFile file : files) {
     if (!file.isFile()) continue;
     pool.submit(new FtpFileDownloader(file));
}

FTPFileDownloader.java


public class FtpFileDownloader implements Runnable{
    static Logger logger = LoggerFactory.getLogger(FtpFileDownloader.class);

    private FTPFile file;

    public FtpFileDownloader(FTPFile file) {
        this.file = file;
    }

    private OutputStream outputStream;

    @Override
    public void run() {
        try{
            long start = System.currentTimeMillis();
            String fileName = file.getName();
            logger.info("File is {}", fileName);

            outputStream = new BufferedOutputStream(new FileOutputStream("/home/user/Downloads/" + "FtpDownloads" + "/" + fileName));

            //get the file from the remote system
            ftpClient.retrieveFile(fileName, outputStream);

            showServerReply(ftpClient);
            logger.info("[{}ms, {} processing finished.]",System.currentTimeMillis()-start,fileName);


        }catch (Exception e){
            logger.info("FtpFileDownloader expection");
            e.printStackTrace();
        }finally {
            try {
                //close output stream
                outputStream.close();
            } catch (IOException e) {
                logger.info("Io exception happened");
                e.printStackTrace();
            }
        }

I've created a fixed thread pool of size 5. So after downloading 5 files from the server by each individual server

Thread is not disconnecting from the server even after its file is downloaded and waiting there for FTP server to disconnect

2022-03-01 15:49:33.584  INFO 10931 --- [pool-1-thread-5] t.a.f.listener.ftp.FtpFileDownloader     : File is mail-send-winforms.png
2022-03-01 15:49:33.587  INFO 10931 --- [pool-1-thread-4] t.a.f.listener.ftp.FtpFileDownloader     : File is mail-editor.png
2022-03-01 15:50:33.769  INFO 10931 --- [pool-1-thread-1] t.a.f.listener.ftp.FtpFileDownloader     : FtpFileDownloader expection
2022-03-01 15:50:33.771  INFO 10931 --- [pool-1-thread-1] t.a.f.listener.ftp.FtpFileDownloader     : File is mime-explorer.png
java.net.SocketTimeoutException: Connect timed out
    at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:546)
    at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:597)
    at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
    at java.base/java.net.Socket.connect(Socket.java:633)
    at org.apache.commons.net.ftp.FTPClient._openDataConnection_(FTPClient.java:866)
    at org.apache.commons.net.ftp.FTPClient._retrieveFile(FTPClient.java:971)
    at org.apache.commons.net.ftp.FTPClient.retrieveFile(FTPClient.java:3308)
    at tech.adoptnet.ftppractice.listener.ftp.FtpFileDownloader.run(FtpFileDownloader.java:35)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:833)

after downloading the files every thread is waiting until connection time out and then it goes to download other files.

How to make thread resuse itself without connection time out?

Are all of your jobs sharing the same `ftpClient`? Are you sure it is reentrant? I would think that each of your downloader threads should have it's own client although that would change the model if each thread wanted to use the same client connection to download multiple files. — Gray, Mar 01 '22 at 20:10
Yes, all of my jobs share the same ```ftpClient``` and that's the reason I think it's happening and yes you are right I need to change the architecture of it. — Malav Mevada, Mar 02 '22 at 05:24

Prachi Patel · Accepted Answer · 2022-03-02T05:44:38.023

0

I think I got what you're trying to do. First of all, FTP doesn't itself let you use multiple connections simultaneously (For ref Uploading files in parallel with Java FTP client).

What you're doing is not thread-safe as well. So I advise you to make parallel connections and download from there.

edited Mar 02 '22 at 05:44

answered Mar 02 '22 at 05:24

Prachi Patel

222
2
9

It's not FTP itself doing that, it is the Apache `FTPClient`. FTP allows multiple connections. I use 4 all the time. – user207421 Mar 02 '22 at 05:30
1

Yeah, my bad I got that from another stack overflow verified answer but my main focus was on thread-safety. So even if it lets you have multiple connections I think it might not be thread-safe for file downloading. – Prachi Patel Mar 02 '22 at 05:42
That answer doesn't say what you claim here. It says 'over one FTP connection', and my own daily usage contradicts your claim. You still need to fix your answer. – user207421 Mar 03 '22 at 01:10
@OP What you need to do is use multiple `FTPClient` objects. This answer is not correct. – user207421 Mar 03 '22 at 02:24

Thread is not disconnecting from server even after its file is downloaded from ftp server and not reusing itself to download another one

1 Answers1