"outOfMemoryError java heap space" calculating histogram, trade of RAM vs Time?

Question

I've been working with my project for a while which at some points calculates the Histogram of a big image (Working with photos of up to 40Mpx, but in general around 10-20Mpx.

I was always using a laptop with 16GB of RAM and I had not noticed any problem. Today I switched to a 6GB ram laptop and with photos of 17Mpx this Exception started to appear when I was calculating an Histogram.

I had switched to this way of calculating it because It was faster than iterating over all pixels and getting all the colors in each pixel.

What is your advice on how should I write such a code?

If I'm looking to make the program faster I think I need to use more RAM (this big double[] object). If the PC has enough RAM there won't be any problem and the program will run smooth, but if the PC doesn't have this much RAM it will just crash and make the program useless.

So should I write the code in a "slower way" by iterating over all pixels by hand and make it "safer"?

Or am I doing something wrong and both things can be done at the same time?

This is the piece of code were this outOfMemoryError occurs:

// dataset
    dataset = new HistogramDataset();
    final int w = image.getWidth();
    final int h = image.getHeight();
    double[] r = new double[w * h]; //Here some PC's with not enough RAM will crash
    double[] s = new double[w * h];
    double[] t;
    r = raster.getSamples(0, 0, w, h, 0, r);
    s = r;
    dataset.addSeries(lang.getString("HistogramRGB.String.red"), r, BINS);
    r = raster.getSamples(0, 0, w, h, 1, r);
    t = new double[r.length + s.length]; //Add R+G
    System.arraycopy(s, 0, t, 0, s.length);
    System.arraycopy(r, 0, t, s.length, r.length);
    dataset.addSeries(lang.getString("HistogramRGB.String.green"), r, BINS);
    r = raster.getSamples(0, 0, w, h, 2, r);
    s = new double[r.length + t.length]; //Add R+G+B
    System.arraycopy(t, 0, s, 0, t.length);
    System.arraycopy(r, 0, s, t.length, r.length);
    dataset.addSeries(lang.getString("HistogramRGB.String.blue"), r, BINS);
    dataset.addSeries(lang.getString("HistogramRGB.String.brigthness"), s, BINS);

    // chart
    chart = ChartFactory.createHistogram(lang.getString("HistogramRGB.String.histogram"), "",
            "", dataset, PlotOrientation.VERTICAL, false, true, false);

Update: Using the option -Xmx suggested in the comments solves the problem.

Results using @TheConstructor optimization, in a virtual machine using windows 10 32bits and 3,5GB ram:

• Before the optimization with less than -Xmx1444m will run the exception
• After the optimization with less than -Xmx824m will run the exception

This is what I have by default:

java -XX:+PrintFlagsFinal -version | findstr HeapSize
uintx ErgoHeapSizeLimit                         = 0                                   {product}
uintx HeapSizePerGCThread                       = 67108864                            {product}
uintx InitialHeapSize                          := 16777216                            {product}
uintx LargePageHeapSizeThreshold                = 134217728                           {product}
uintx MaxHeapSize                              := 268435456                           {product}
java version "1.8.0_111"
Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
Java HotSpot(TM) Client VM (build 25.111-b14, mixed mode, sharing)

Which is around 268MB, and the maximum I can set by the command in this computer is 1,5GB. I find it strange that without anything else opened the whole windows with any other program requires 2GB of the 3.5GB.

Answer 1

Ultimately I guess that you will need to specify a correctly sized -Xmx or -XX:MaxHeapSize argument to your java-call. The defaults are derived from available memory and limit the amount of memory Java can use. Try to figure out a working size. You could try e.g. -Xmx2g. Some details on -Xmx can be found inside documentation

Looking at your code you can eliminate t and skip initialization of s. Though I guess it will not solve all issues here are my modifications:

    // dataset
    dataset = new HistogramDataset();
    final int w = image.getWidth();
    final int h = image.getHeight();
    double[] buffer = new double[w * h];
    double[] rgb;

    buffer = raster.getSamples(0, 0, w, h, 0, buffer);
    rgb = Arrays.copyOf(buffer, buffer.length * 3); // copy as otherwise it gets overwritten in next getSamples
    dataset.addSeries(lang.getString("HistogramRGB.String.red"), buffer, BINS);

    buffer = raster.getSamples(0, 0, w, h, 1, buffer);
    System.arraycopy(buffer, 0, rgb, buffer.length, buffer.length); //Add G
    dataset.addSeries(lang.getString("HistogramRGB.String.green"), buffer, BINS);

    buffer = raster.getSamples(0, 0, w, h, 2, buffer);
    System.arraycopy(buffer, 0, rgb, buffer.length * 2, buffer.length); //Add B
    dataset.addSeries(lang.getString("HistogramRGB.String.blue"), buffer, BINS);

    dataset.addSeries(lang.getString("HistogramRGB.String.brigthness"), rgb, BINS);

    // chart
    chart = ChartFactory.createHistogram(lang.getString("HistogramRGB.String.histogram"), "", "", dataset,
            PlotOrientation.VERTICAL, false, true, false);

Depending on whether or not addSeries creates a copy of the supplied data, you will probably need to assign buffer a new array before each call to getSamples. If I correctly guessed it is Raster#getSamples you could also use (double[]) null as argument instead of buffer and let getSamples allocate the array for you.

If precision doesn't matter too much you could also switch double[] with float[] which saves half of the memory.

Answer 2

If possible it is better to reduce the objects creation so that heap space memory will be reduced. If all the objects are mandatory in your application, then use command line arguments while running your application:

java -Xms<size>        set initial Java heap size 

(OR)

java -Xmx<size>        set maximum Java heap size