Reading multiple files in loop from FTP server using Apache Commons Net FTPClient

Question

I have list of files that needs to be read from FTP server. I have a method readFile(String path, FTPClient client) which reads and prints the file.

public byte[] readFile(String path,FTPClient client){
    InputStream inStream = null;
    ByteArrayOutputStream os = null;
    byte[] finalBytes = new byte[0];
            int reply;
    int len;
    byte[] buffer = new byte[1024];
    try{
        os = new ByteArrayOutputStream();
        inStream = client.retrieveFileStream(path);
        reply = client.getReplyCode();
        log.warn("In getFTPfilebytes() :: Reply code -"+reply);

        while ((len = inStream.read(buffer)) != -1) {
                    // write bytes from the buffer into output stream
                    os.write(buffer, 0, len);
                }
        finalBytes = os.toByteArray();

    if(inStream == null){
        throw new Exception("File not found");
    }

    inStream.close();
    }catch(Exception e){

    }finally{
        try{ inStream.close();} catch(Exception e){}
    }
    return finalBytes;

}

I am calling above method in loop of list which contains strings of file path.

Issue - In loop only first file is getting read properly. Afterwards, it does not read file and throws an exception. inStream gives NULL for second iteration/second file. Also while iterating first file reply code after retrieveFileStream is "125(Data connection already open; transfer starting.)"

In second iteration it gives "200 (The requested action has been successfully completed.)"

I am not able to understand what is wrong here. Have not closing inputstream connection properly?

Martin Prikryl · Answer 1 · 2021-09-07T10:48:29.773

You have to call FTPClient.completePendingCommand and close the input stream, as the documentation for FTPClient.retrieveFileStream says:

Returns an InputStream from which a named file from the server can be read. If the current file type is ASCII, the returned InputStream will convert line separators in the file to the local representation. You must close the InputStream when you finish reading from it. The InputStream itself will take care of closing the parent data connection socket upon being closed.

To finalize the file transfer you must call completePendingCommand and check its return value to verify success. If this is not done, subsequent commands may behave unexpectedly.

inStream = client.retrieveFileStream(path);
try {
    while ((len = inStream.read(buffer)) != -1) {
        // write bytes from the buffer into output stream
        os.write(buffer, 0, len);
    }
    finalBytes = os.toByteArray();
} finally {
    inStream.close()
    if (!client.completePendingCommand()) {
        // error
    }
}

Btw, there are better ways for copying from InputStream to OutputStream:
Easy way to write contents of a Java InputStream to an OutputStream

score 1 · Answer 2 · answered Sep 30 '19 at 11:46

1

According to the documentation of the FTPClient.retrieveFileStream() method,

You must close the InputStream when you finish reading from it. The InputStream itself will take care of closing the parent data connection socket upon being closed.

When you close the stream, your client connection will be closed too. So instead of using the same client over and over, you need to create a new client connection for each file.

answered Sep 30 '19 at 11:46

Udith Gunaratna

2,091
1
13
17

I have to pass FTP parameters to the method to create FTPClient again and again. So you are saying that by anyhow i cant use the same client for further processing ? – Nitin Rathod Sep 30 '19 at 12:44

score -1 · Answer 3 · answered Sep 30 '19 at 11:50

-1

I didn't see the output stream is not properly closed.
finalBytes is o bytes?
where you defined the buffer variable?

please log the path so that we can see the path is correct or not. I guess the stream which is not properly closed makes the issue

answered Sep 30 '19 at 11:50

Mohamed Ismail M

65
8

I will try closing OutputStream and check if it is working – Nitin Rathod Sep 30 '19 at 12:41

Reading multiple files in loop from FTP server using Apache Commons Net FTPClient

3 Answers3