Upload file in Android with outofmemory error

Question

My upload code as below:

String end = "\r\n";
String twoHyphens = "--";
String boundary = "*****";
try {
URL url = new URL(ActionUrl);
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setDoInput(true);
con.setDoOutput(true);
con.setUseCaches(false);
con.setRequestMethod("POST");
con.setRequestProperty("Connection", "Keep-Alive");
con.setRequestProperty("Accept", "text/*");
con.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);
DataOutputStream ds = new DataOutputStream(con.getOutputStream());
ds.writeBytes(twoHyphens + boundary + end);
ds.writeBytes("Content-Disposition: form-data;" + "name=\"folder\"" + end + end);
ds.write(SavePath.getBytes("UTF-8"));
ds.writeBytes(end);
ds.writeBytes(twoHyphens + boundary + end);
ds.writeBytes("Content-Disposition: form-data;" + "name=\"Filedata\"; filename=\"");
ds.write(FileName.getBytes("UTF-8"));
ds.writeBytes("\"" + end);
ds.writeBytes(end);
FileInputStream fStream = new FileInputStream(uploadFilepath+""+FileName);
int bufferSize = 1024;
byte[] buffer = new byte[bufferSize];
int length = -1;
int pro = 0;
while((length = fStream.read(buffer)) != -1) {
ds.write(buffer, 0, length);
}       
ds.writeBytes(end);
ds.writeBytes(twoHyphens + boundary + twoHyphens + end);
fStream.close();
ds.flush();
InputStream is = con.getInputStream();
int ch;
StringBuffer b = new StringBuffer();
while((ch = is.read()) != -1) {
b.append((char)ch);
}
ds.close();
}
catch(Exception e) {
e.printStackTrace();
}

It can works while upload a smaller file. But while more 16 mb, it will upload fail and show the OutOfMemory error. I think it cause by put all data in buffer. So I want to make it to send data while buffer save 1024 bytes. But I no idea to do that. May anyone help me to do it?

Answer 1

brian, you should add

con.setChunkedStreamingMode(0);

before

DataOutputStream ds = new DataOutputStream(con.getOutputStream());

if your server could support chunked mode, or add

con.setFixedLengthStreamingMode(packet_size);

where packet_size = upload_file_size + header_size.

Answer 2

You should confirm at what point your error occurs. I suspect that it's during reading the response. In this case, it seems that server may be responding with a lot of data that you place in the StringBuffer. Do you actually need to consume the entire response and keep it in memory? If it's a file, save it rather than keeping in memory.

---

I did some more research and here is one other possibility. Android JVM by default has 16mb max heap. See this for some details.

On the other hand, if your server does not actually consume the data, most of it will reside on the client. So if you have more than max heap of data the client will fail.

So I suspect that your server just does not read the data from the stream.

The following class (which is a snippet of relevant parts of your code) illustrates the problem. Run it on any JVM like following:

java -Xmx16m -cp . Test

and it will produce OOM very quickly. In fact, much earlier than expected.

import java.io.*;
import java.net.*;

public class Test {
  public static void main(String argv[]) throws Exception {
    new Thread() {
      public void run() {
        try {
           new ServerSocket(12000).accept();
        } catch (Throwable t) {
          t.printStackTrace();
        } 
      }
    }.start();

    URL url = new URL("http://localhost:12000/");
    HttpURLConnection con = (HttpURLConnection) url.openConnection();
    con.setDoInput(true);
    con.setDoOutput(true);
    con.setUseCaches(false);
    DataOutputStream ds = new DataOutputStream(con.getOutputStream());
    int bufferSize = 1024;
    byte[] buffer = new byte[bufferSize];
    for (int i=0;i<100000;i++) {
      ds.write(buffer, 0, 1024);
      System.out.println("Written chunk " + i);
    }       
    ds.flush();
  }
}