Resume file upload/download after lost connection (Socket programming)

Question

I'm writing a program to download/upload a file between a client and server using socket programming. The code i've written till now works in the sense that i can sucesfully transfer files. However , if a connection fails due to problem in the network/client/server while a download / upload is occuring.. i need to RESUME the download/upload from the original point(Do not want the originally sent data to be resent). I'm not sure how to go about this. I'm reading the file into a byte array and sending it across the network. My initial idea is that everytime i'm downloading.. i should check if the file already exists and read the data into a byte array --> send the data to the server for comparison and then return the remaining data from the server file by comparing the two byte arrays. But this seems inefficient and takes away the point of resuming a download(since i'm sending the data again). Note: The file name is an unique identifier. I would really appreciate it if anybody could give me suggestions as to how i should implement the file resume functionality?

Server side code:
    package servers;
    import java.io.*;
    import java.net.*;
    import java.util.Arrays;
    public class tcpserver1 extends Thread
    {
        public static void main(String args[]) throws Exception 
    {
        ServerSocket welcomeSocket = null;
        try
        {
             welcomeSocket = new ServerSocket(5555);
             while(true)
             {
                 Socket socketConnection = welcomeSocket.accept();
                 System.out.println("Server passing off to thread");
                 tcprunnable tcprunthread = new tcprunnable(socketConnection);
                 Thread thrd = new Thread(tcprunthread);
                 thrd.start();
                 System.out.println(thrd.getName());
             }
        }
         catch(IOException e){
             welcomeSocket.close();
             System.out.println("Could not connect...");
         }
      }
}
class tcprunnable implements Runnable
{
    Socket socke;
    public tcprunnable(Socket sc){
         socke = sc;
    }


    public void download_server(String file_name)
    {   
        System.out.println("Inside server download method");
        try
        {
        System.out.println("Socket port:" + socke.getPort());

        //System.out.println("Inside download method of thread:clientsentence is:"+clientSentence);
        // Create & attach output stream to new socket
        OutputStream outToClient = socke.getOutputStream();
        // The file name needs to come from the client which will be put in here below
        File myfile = new File("D:\\ "+file_name);
        byte[] mybytearray = new byte[(int) myfile.length()];
        BufferedInputStream bis = new BufferedInputStream(new FileInputStream(myfile));
        bis.read(mybytearray, 0, mybytearray.length);
        outToClient.write(mybytearray, 0, mybytearray.length);
        System.out.println("Arrays on server:"+Arrays.toString(mybytearray));
        outToClient.flush();
        bis.close();    
    }
        catch(FileNotFoundException f){f.printStackTrace();}
        catch(IOException ie){
            ie.printStackTrace();
        }
    }

    public void upload_server(String file_name){

        try{

        byte[] mybytearray = new byte[1024];
            InputStream is = socke.getInputStream();
            ByteArrayOutputStream baos = new ByteArrayOutputStream();
            FileOutputStream fos = new FileOutputStream("D:\\ "+file_name);
            BufferedOutputStream bos = new BufferedOutputStream(fos);
            int bytesRead = is.read(mybytearray, 0, mybytearray.length);
                    bos.write(mybytearray, 0, bytesRead);
            do {
                baos.write(mybytearray);
                bytesRead = is.read(mybytearray);
        }
            while (bytesRead != -1);
            bos.write(baos.toByteArray());
            System.out.println("Array on server while downloading:"+Arrays.toString(baos.toByteArray()));
            bos.close();
    }
        catch(FileNotFoundException fe){fe.printStackTrace();}
        catch(IOException ie){ie.printStackTrace();}


    }

    @Override
    public void run()
    {   
        try
        {       
            System.out.println("Server1 up and running" + socke.getPort());
            //  Create & attach input stream to new socket
                BufferedReader inFromClient = new BufferedReader
                (new InputStreamReader(socke.getInputStream()));
                // Read from socket
                String  clientSentence = inFromClient.readLine();
                String file_name = inFromClient.readLine();

                System.out.println("Sever side filename:" + file_name);
                try{
                if(clientSentence.equals("download"))
                {
                    download_server(file_name);
                }
                else if(clientSentence.equals("upload"))
                {

                    upload_server(file_name);
                    System.out.println("Sever side filename:" + file_name);
                }
                else
                {
                    System.out.println("Invalid input");
                }
                }
                catch(NullPointerException npe){
                    System.out.println("Invalid input!");
                }
                    socke.close();  
        }
        catch(IOException e)
        {
            e.printStackTrace();
            System.out.println("Exception caught");
        }           
    }
}

Client side code:

package clients;
import java.io.*;
import java.net.*;
import java.util.Arrays;
public class tcpclient1 
{   
        public static void main (String args[]) throws Exception
        {   
                // Create input stream to send sentence to server
                BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in));
                Socket clientSocket = null;
                while(true){
                System.out.println("Please enter the server you want to use");
                System.out.println("Enter 1 for Server 1 and 2 for Server2");
                String server_choice = inFromUser.readLine();

                if(server_choice.equals("1")){
                // Create client socket to connect to server
                // The server to use will be specified by the user
                 clientSocket = new Socket("localhost",5555);
                 break;
                }
                else if(server_choice.equals("2"))
                {
                    clientSocket = new Socket("localhost",5556);
                    break;
                }
                else
                {
                    System.out.println("Invalid entry");
                }
        }


                System.out.println("Please enter download for dowloading");
                System.out.println("Please enter upload for uploading");

            // sentence is what'll be received from input jsp   
                String sentence = inFromUser.readLine();

                if(sentence.equals("download"))
                {
                    download_client(clientSocket,sentence);
                }
                else if(sentence.equals("upload"))
                {
                    upload_client(clientSocket,sentence);
                }
                else
                {
                    System.out.println("Invalid input");
                }

            clientSocket.close();
        }

        public static void download_client(Socket clientSocket , String sentence)
        {
            try{
            // Create output stream attached to socket
            DataOutputStream outToServer = new DataOutputStream(clientSocket.getOutputStream());

            // Send line to server
            outToServer.writeBytes(sentence+'\n');
            BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in));
            System.out.println("Enter the name of file to download:");
            String file_to_download = inFromUser.readLine();


            if(searching(file_to_download))
            {

                // Read local file and send that to the server for comparison
            // DONT THINK THIS IS THE RIGHT WAY TO GO ABOUT THINGS SINCE IT BEATS THE PURPOSE OF RESUMING A DOWNLOAD/UPLOAD 

            }


            // Send filetodownload to server
            outToServer.writeBytes(file_to_download+'\n');
            byte[] mybytearray = new byte[1024];
            InputStream is = clientSocket.getInputStream();
            ByteArrayOutputStream baos = new ByteArrayOutputStream();

            FileOutputStream fos = new FileOutputStream("E:\\ "+file_to_download);
            BufferedOutputStream bos = new BufferedOutputStream(fos);
            int bytesRead = is.read(mybytearray, 0, mybytearray.length);
            bos.write(mybytearray, 0, bytesRead);
            do {
                baos.write(mybytearray);
                bytesRead = is.read(mybytearray);
            }
            while (bytesRead != -1);
            bos.write(baos.toByteArray());
            System.out.println("Array on client while downloading:"+Arrays.toString(baos.toByteArray()));
            bos.close();    
      }
            catch(FileNotFoundException fe){fe.printStackTrace();}
            catch(IOException ie){ie.printStackTrace();}

        }
        public static void upload_client(Socket clientSocket, String sentence)
        {
                try{
                    // Create output stream attached to socket
                DataOutputStream outToServer1 = new DataOutputStream(clientSocket.getOutputStream());
                    // Send line to server
                outToServer1.writeBytes(sentence+'\n');
                    System.out.println("In the client upload method");

                    BufferedReader inFromUser = new BufferedReader(new InputStreamReader(System.in));
                    System.out.println("Enter the name of file to upload:");
                    String file_to_upload = inFromUser.readLine();
                    //System.out.println("Cline side file name:"+file_to_upload);
                    outToServer1.writeBytes(file_to_upload+'\n');
                    System.out.println(file_to_upload);
                    OutputStream outtoserver = clientSocket.getOutputStream();

            File myfile = new File("E:\\ "+file_to_upload);
            byte[] mybytearray = new byte[(int) myfile.length()];
            BufferedInputStream bis = new BufferedInputStream(new FileInputStream(myfile));
            bis.read(mybytearray, 0, mybytearray.length);
            outtoserver.write(mybytearray, 0, mybytearray.length);

            System.out.println("filename:"+file_to_upload+"Arrays on client while uploading:"+Arrays.toString(mybytearray));
            outtoserver.flush();
            bis.close();    
                }
                catch(FileNotFoundException fe){fe.printStackTrace();}
                catch(IOException ie){ie.printStackTrace();}
        }


        public static boolean searching(String file_name)
        {
            String file_path = "E:\\ "+file_name;
            File f = new File(file_path);
            if(f.exists() && !f.isDirectory()) { return true; }
            else
                return false;

        }       
       }

The above code runs fine for transferring files between the client and server.
Again , would really appreciate any help!

Just a thought: you might want to add a `java` tag to your question, for example by replacing "file". That would more likely get the attention of the people who can help you. — cnicutar, Feb 28 '14 at 22:58
I hadn't removed some of the comments in my code before posting it. Have done that now. It works fine for download/upload. I'm wondering how to implement the resume download/upload of a file functionality?(i.e If the client has downloaded 30kb of data before connection failed.. the next time it should not download those 30kb's). Where and how should i use the result of read? — user3343547, Feb 28 '14 at 23:19
An efficient implementation of resuming downloads is a protocol with support for file information; for instance, FTP has `MDTM`, HTTP has `Cache-Control`, etc etc. "Blank" refetching from the same location, for instance, won't prevent you from downloading from _another file_ at the same location on a resume... FTP and HTTP avoid that pitfall. — fge, Feb 28 '14 at 23:45

score 8 · Accepted Answer · edited May 23 '17 at 12:25

There are many ways which you can do this, I suggest you to create a separate type of request to the server that accepts the file's name and file position which is the position where in the file where the connection failed.

That's how you will get the file from the server in the client's side:

int filePosition = 0;

InputStream is = clientSocket.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();

do {
    baos.write(mybytearray);
    bytesRead = is.read(mybytearray);

    if(bytesRead != -1)
        filePosition += bytesRead;
}
while (bytesRead != -1);

Now if the connection got interrupted for some reason you can send a request again to the server with the same file name and the filePosition, and the server will send the file back like this:

OutputStream outToClient = socke.getOutputStream();
// The file name needs to come from the client which will be put in here below
File myfile = new File("D:\\ "+file_name);
byte[] mybytearray = new byte[(int) myfile.length()];
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(myfile));
bis.skip(filePosition) //Advance the stream to the desired location in the file
bis.read(mybytearray, 0, mybytearray.length);
outToClient.write(mybytearray, 0, mybytearray.length);
System.out.println("Arrays on server:"+Arrays.toString(mybytearray));
outToClient.flush();
bis.close();

And in the client you can open the file stream and specify append = true in the constructor like this:

FileOutputStream fos = new FileOutputStream("D:\\ "+file_name, true);

This could be one way to do this, there are a lot more options. And I also suggest verify the files after the transfer using some hash function like MD5 for example, it creates unique stamp for a given input and it always outputs same result for the same input, which means, you can create the stamp from the same file both in the server and in the client and if the file is truly the same, it will generate the same stamp. Since the stamp's size is very small relative to the file it self and it is also fixed, it can be send between the client/server without much overhead.

You can generate an MD5 hash with this code:

MessageDigest md = MessageDigest.getInstance("MD5");
try (InputStream is = Files.newInputStream(Paths.get("file.txt"))) {
  DigestInputStream dis = new DigestInputStream(is, md);
  /* Read stream to EOF as normal... */
}
byte[] digest = md.digest();

(taken from: Getting a File's MD5 Checksum in Java)

score 2 · Answer 2 · answered Feb 28 '14 at 23:32

2

Basically, when requesting a download You should attach information about how many bytes need to be skipped (0 on new download). You should get this information from part of the file that you have downloaded (read it's size). Server should skip given count of bytes and send back the remainder of file. Client should append this to the existing file. For sanity check, You could add some file hash checking in the end, to ensure You got the file correctly.

answered Feb 28 '14 at 23:32

endriu_l

1,649
16
20

That's exactly what I just wrote in my answer, I didn't saw your answer until I was done writing mine :) – UnTraDe Feb 28 '14 at 23:35
No problem, question was about suggestions, so I didn't bother with implementation. Just as much help as needed for own experiments :) . In the end I guess Your answer is more useful – endriu_l Feb 28 '14 at 23:38
I meant that it gave me more confidence when your answer used the same method as mine :D – UnTraDe Feb 28 '14 at 23:42

Resume file upload/download after lost connection (Socket programming)

2 Answers2

Linked

Related