OutOfMemoryError at java.net.URLEncoder.encode

Question

I want to encode a Base64 encoded string using URLEncoder.encode, this string contains the image, it may be of any size (1MB-8MB).

String imageDataToUpload = URLEncoder.encode(temp, "UTF-8");

02-27 02:41:45.213: D/dalvikvm(18600): GC_FOR_ALLOC freed 1824K, 12% free 26452K/29831K, paused 161ms
02-27 02:41:45.213: I/dalvikvm-heap(18600): Forcing collection of SoftReferences for 2968580-byte allocation
02-27 02:41:45.383: D/dalvikvm(18600): GC_BEFORE_OOM freed 9K, 12% free 26443K/29831K, paused 138ms
02-27 02:41:45.383: E/dalvikvm-heap(18600): Out of memory on a 2968580-byte allocation.

even i tried this thing in chunks but then StringBuffer.append making the OutOfMemoryError

private String encodeChunckByChunck(String str, final int chunkSize) {
    StringBuffer buffer = new StringBuffer();
    final int size = str.length();
    try {
        for (int i = 0; i < size; i += chunkSize) {
            Log.d("Encode", "..........inside loop");
            if (i + chunkSize < size) {
                Log.d("Encode", "..........full chunk");
                buffer.append(URLEncoder.encode(str.substring(i, i + chunkSize), "UTF-8"));
            } else {
                Log.d("Encode", "..........half chunk");
                buffer.append(URLEncoder.encode(str.substring(i, size), "UTF-8"));
            }
        }
    } catch (UnsupportedEncodingException e) {
        Log.d("Encode", "..........exception:" + e.getMessage());
        e.printStackTrace();
    }
    Log.d("Encode", "..........before returning function");
    return buffer.toString();
}

That's basically prohibitively large. http://stackoverflow.com/q/2881139/139010 — Matt Ball, Feb 26 '13 at 21:34
@MattBall then how can i write this on OutputStream, actually i've to write URLEncode String. Can you please suggest anything. — umesh, Feb 26 '13 at 21:49
That's a really big chunk of data to send in a URL. Why not post the raw bytes to the server? — tom, Feb 26 '13 at 22:07

score 0 · Answer 1 · answered Feb 26 '13 at 22:04

0

Can you write it to a file, piece by piece instead of storing it in memory? Otherwise, use less memory (free up some large variables) elsewhere in your app.

answered Feb 26 '13 at 22:04

EricFromChina

113
11

score 0 · Answer 2 · edited May 23 '17 at 11:57

Assuming that (as others already pointed out, and I agree) it's a memory problem, if I were you I'd implement custom OutputStream which would be doing URLEncoding on the fly, on a small chunks of data.

It could look like this (partial code, you'd have to fill it in order to compile):

public static class UrlEncOutputStream extends FileOutputStream {

    /* Implement constructors here */

    @Override
    public void write (byte [] b, int off, int len)  throws IOException {
            String s = StringUtils.newStringUsAscii(b);
            super.write(URLEncoder.encode(s, "UTF-8").getBytes());
    }

    /* Implement write(byte [] b) and write(int b) in similar fashion */
}

You can also (depending on your use case) avoid loading whole file to the memory at once, by using encode() method posted by Tomasz Nurkiewicz here:

public void encode(File file, OutputStream base64OutputStream) {
  InputStream is = new FileInputStream(file);
  OutputStream out = new Base64OutputStream(base64OutputStream)
  IOUtils.copy(is, out);
  is.close();
  out.close();
}

Now you have everything in place, so you can invoke it:

UrlEncOutputStream out = new UrlEncOutputStream("OUT.txt");
encode(new File("SOURCE.IMG"), out);

Hope this helps!

Edit: In order for this code to work, you need to have commons-codec and commons-io in your classpath.

OutOfMemoryError at java.net.URLEncoder.encode

2 Answers2