How to transfer a file from url to ftp using java?

Question

Given a url, I want to transfer the file in that url to a specific to ftp. I have two options: 1) Using external library (like apache common-io) I can fetch the file from the url and then upload it to the ftp. 2) Open two sockets (to the url an to the ftp) and simply transfer the file from the url to the ftp.

I'm I right about this? What's the best way to do that?

I'm using spring.

Here are some links that I found about it:

How to download and save a file from Internet using Java?.

Here is the apache common-io library

Edit:

I'm dealing with 100-200M files.

Update: I choose to first download the file and then upload it. I choose to do this since I don't expect many problems downloading the file, but I expect problems uploading the file to the FTP server. So, I think the the error handling will better that way. The lack of this solution is the fact that I'm storing the temporary file in the disk. I'll update later if I'll encounter some problems.

score 4 · Answer 1 · answered Jul 23 '12 at 17:40

1. Frankly Speaking, If you are using Open-Source, then take the advantage of External Libraries.

2. Do Not re-invent the wheel, you have the trusted libraries at your disposal, so use it.

I am also attaching my program which i used to upload and download song to ftp server using the apache's common lib

Uploading :

public void goforIt(){


        FTPClient con = null;

        try
        {
            con = new FTPClient();
            con.connect("192.168.2.57");

            if (con.login("Administrator", "KUjWbk"))
            {
                con.enterLocalPassiveMode(); // important!
                con.setFileType(FTP.BINARY_FILE_TYPE);
                String data = "/sdcard/vivekm4a.m4a";

                FileInputStream in = new FileInputStream(new File(data));
                boolean result = con.storeFile("/vivekm4a.m4a", in);
                in.close();
                if (result) Log.v("upload result", "succeeded");
                con.logout();
                con.disconnect();
            }
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }






    }

Dowloading:

public void goforIt(){
    FTPClient con = null;

    try
    {
        con = new FTPClient();
        con.connect("192.168.2.57");

        if (con.login("Administrator", "KUjWbk"))
        {
            con.enterLocalPassiveMode(); // important!
            con.setFileType(FTP.BINARY_FILE_TYPE);
            String data = "/sdcard/vivekm4a.m4a";

            OutputStream out = new FileOutputStream(new File(data));
            boolean result = con.retrieveFile("vivekm4a.m4a", out);
            out.close();
            if (result) Log.v("download result", "succeeded");
            con.logout();
            con.disconnect();
        }
    }
    catch (Exception e)
    {
        Log.v("download result","failed");
        e.printStackTrace();
    }



}

score 2 · Accepted Answer · answered Jul 23 '12 at 16:37

Would require you to read the entire file referenced by the URL before you can send it to the ftp server. If the file is really large, you may need to write it to a temporary file when you read from the URL connection, then read from the temporary file when you send to the FTP server. With this method, keeping the reading from the URL and the writing to the FTP server separate, you can handle the errors separately. For example, if the URL connection throws exceptions, you can handle that accordingly, and perhaps retry the request. You can ensure you have the complete file before any communication with the FTP server is attempted.
You can do this if you've established the input and output streams and are ready to read from and write to them. This means you've already done all the protocol stull, and on the FTP server, that means telling it that you want to upload a file, where you want to upload it, and what the file's name is. Then you can simply read from the URL input stream and write that block to the FTP's output stream. Because this is streamed, if the communication between you and the webserver, or you and the FTP server gets interrupted. You'll probably have to restart everything from the beginning.

Either way is acceptable.

Thanks! Your detailed answer really helped me, I think I'll go with the first way since I'm dealing with 100-200M files and losing the connection can be very expensive... Since I'll need to star all over again even if the problem is in the FTP server. — Noam, Jul 23 '12 at 19:30
When you are working with 2 open stream you can make retry only to the small ammount of data (buf, buffer). Download connection will be open for a long time (you will wait for uploading), so your connection can be droped. If you don't have ability to continie download from the offset, you will have to start all the process all over again. — alexsmail, Jul 24 '12 at 08:22

score 0 · Answer 3 · answered Mar 21 '13 at 11:00

IMPORTS :
import org.apache.commons.net.ftp.FTP;
import org.apache.commons.net.ftp.FTPClient;

CODE : 
public static void main(String args[])
 try {
   //Create a FTP Client
   FTPClient ftp = new FTPClient();

  //Set Host Information
   ftp.connect(HOST_NAME);
   ftp.login(USER_NAME, PASSW)RD);

  //Set File Type "Specially for PDF's"
   ftp.setFileType(FTP.BINARY_FILE_TYPE, FTP.BINARY_FILE_TYPE);
   ftp.setFileTransferMode(FTP.BINARY_FILE_TYPE);

  //Change the current Directory to a desired one
   ftp.changeWorkingDirectory(SOME_DIRECTORY);

  //get File from the FTP location
   String fileNames[] = ftp.listNames();
   List<String> fTPFileNameList = new ArrayList<String>();

  //Iterate to get File Names
   if (fileNames != null && fileNames.length > 0) {
    for (String fName : fileNames) {
     fTPFileNameList.add((new File(fName)).getName());
    }
   }

  // Check the file is present on FTP
   if(fileNames != null && fileNames.length > 0 && fTPFileNameList.contains(FILE_NAME)){
    //File is present on FTP
   } else {
    //File is not present on FTP
    String httpURL = "YourDesiredURL";
    InputStream fis;
    try{
     fis = new URL(httpURL).openStream();
    } catch (FileNotFoundException e) {
     //File is not present on the URL
    }
    ftp.storeFile(FILE_NAME, fis);
    fis.close();
   }
   ftp.logout();
  } catch (Exception e) {
    e.printStackTrace();
   } 
  }

MAVEN DEPENDENCY : 

< dependency>
        < groupId>commons-net< /groupId>
        < artifactId>commons-net< /artifactId>
        < version>1.3.0</ version>
< /dependency>


JAR LOCATION : 
http://www.jarfinder.com/index.php/java/info/org.apache.commons.net.ftp.FTPClient

How to transfer a file from url to ftp using java?

3 Answers3