Memory issue when storing images in byteArray

Question

I have an app that needs to access a large number of images very quickly, so I need to load those images into memory in some way. Doing so as bitmaps used over 100MB of RAM, which was completely out of the question, so I opted to read jpg files into memory, storing them inside a byteArray. Then I decode them and write them to the canvas as each is needed. This works pretty well, cutting out the slow disk access, while also respecting memory limits.

However, memory usage seems 'off' to me. I'm storing 450 jpgs with a file size of approximately 33kb each. This totals around 15MB of data. However, the app continually runs at between 35MB and 40MB of RAM as reported by both Eclipse DDMS and Android (on a physical device). I've tried modifying how many jpgs are loaded and the RAM used by the app tends to decrease by around 60-70kb per jpg, indicating that each image is stored twice in RAM. Memory usage does not fluctuate which implies that there is not an actual 'leak' involved.

Here is the relevant loading code:

private byte[][] bitmapArray = new byte[totalFrames][];
for (int x=0; x<totalFrames; x++) {
    File file = null;
    if (cWidth <= cHeight){
            file = new File(directory + "/f"+x+".jpg");
    } else {
            file = new File(directory + "/f"+x+"-land.jpg");
    }
    bitmapArray[x] = getBytesFromFile(file);
    imagesLoaded = x + 1;
}


public byte[] getBytesFromFile(File file) {
    byte[] bytes = null;
    try {

        InputStream is = new FileInputStream(file);
        long length = file.length();

        bytes = new byte[(int) length];

        int offset = 0;
        int numRead = 0;
        while (offset < bytes.length && (numRead = is.read(bytes, offset, bytes.length - offset)) >= 0) {
            offset += numRead;
        }

        if (offset < bytes.length) {
            throw new IOException("Could not completely read file " + file.getName());
        }

        is.close();
    } catch (IOException e) {
                  //TODO Write your catch method here
    }
    return bytes;
}

Eventually, they get written to screen like so:

SurfaceHolder holder = getSurfaceHolder();
Canvas c = null;
try {
    c = holder.lockCanvas();
    if (c != null) {            
        int canvasWidth = c.getWidth();
        int canvasHeight = c.getHeight();
        Rect destinationRect = new Rect();
        destinationRect.set(0, 0, canvasWidth, canvasHeight);
        c.drawBitmap(BitmapFactory.decodeByteArray(bitmapArray[bgcycle], 0, bitmapArray[bgcycle].length), null, destinationRect, null);
    }
} finally {
    if (c != null)
    holder.unlockCanvasAndPost(c);
}

Am I correct that there is some sort of duplication going on here? Or is there just that much overhead involved in storing jpgs in a byteArray like this?

Answer 1

Storing bytes in RAM is very different to storing data on hard drives... There is alot more overhead to it. The references to the objects as well the byte array structures all take up additional memory. There isn't really a single source to all the additional memory but just remember than loading a file into RAM normally takes up 2 ~ 3x more space (from experience, I'm afraid I can't quote any documentation here).

Consider this:

File F = //Some file here (Less than 2 GB please)
FileInputStream fIn = new FileInputStream(F);
ByteArrayOutputStream bOut = new ByteArrayOutputStream(((int)F.length()) + 1);

int r;
byte[] buf = new byte[32 * 1000];

while((r = fIn.read(buf) != -1){
    bOut.write(buf, 0, r);
}

//Do a memory measurement at this point. You'll see your using nearly 3x the memory in RAM compared to the file.
//If your actually gonna try this, remember to surround with try-catch and close the streams as appropriate.

Also remember that unused memory is not instantly cleared up. The method getBytesFromFile() may be returning a copy of a byte array which causes memory duplication which may not immediately be garbage collected. If you want to be safe, check the method getBytesFromFile(file) is not leaking any references that should be cleaned up. It won't appear as a memory leak as you only call it a finite number of times.

Answer 2

It might be because your byte array is 2 dimensional, you only need one dimension for loading an image using a byte array, and the second dimension could potentially double the Ram needed as for each byte you would have an empty but still existing byte that you don't use