My method is for some reason using up memory even after it finishes

Question

I was programming a neural network in which I save the data of the weights in one long txt file and so to retrieve that data I used '|' to separate numbers and then added into an array before returning it.

Using this method my ram usage goes to 1500 MB and doesn't go away until the whole program ends but without it my ram usage goes to 700.

I tried to close everything, maybe thinking one of my objects is a thread, but that didn't work either. Is there anything in my code that could be causing this ram usage

Here's the code :

public static int[] findDoub(String fileName) {
    ArrayList<Integer> innt = new ArrayList<Integer>();
    try {
        File file = new File(fileName);
        FileInputStream strem = new FileInputStream(file);
        BufferedInputStream buff = new BufferedInputStream(strem);
        
        int index = buff.read();
        StringBuilder doubus = new StringBuilder("");
        for (int i = 0; index != -1; i++) {
            char a = (char) index;
            if (i > 0) {
                if (a != '|') {
                    doubus.append(a);
                } else {
                    innt.add(Integer.valueOf(doubus.toString()));
                    doubus = new StringBuilder("");
                }
            }
            index = buff.read();
        }
        buff.close();
        strem.close();
        buff = null;
        strem = null;
        
    } catch (IOException e) {
        e.printStackTrace();
    }
    Runtime.getRuntime().gc();
    int[] innnt = new int[innt.size()];
    for (int i = 0; i < innnt.length; i++) {
        innnt[i] = innt.get(i);
    }
    return innnt;
}

You are returning this Object `int[] innnt = new int[innt.size()];` — Scary Wombat, Jun 29 '22 at 00:55
@ScaryWombat I did that on purpose as a way to see if there was something wrong with the Arraylist. With or without the Arraylist, the method for some reason will still take up memory after it finishes being executed. I've had this problem beofre I added the int[] innnt variable — Juan_Puthr, Jun 29 '22 at 01:06
Once your application requires more memory the JVM will allocate it as required, but when the method is complete it doesn't just free it back to the operating system, there is no harm in it remaining allocated to Java until the OS request it back if it needs it. Obviously, the method can be optimized, but it will use memory either way, but is this memory usage causing any problems? And how are you measuring it? — sorifiend, Jun 29 '22 at 01:18
@sorifiend so it isn't actually using that ram space it just has it allocated since it was allocated before and would keep it just in case. I was measuring it using task manager and I kept seeing red which was bothering me. Is there any was to have the JVM remove any unnecessary memory with the Runtime Class or anything? — Juan_Puthr, Jun 29 '22 at 02:07
Yes, take a look here [Does GC release back memory to OS?](https://stackoverflow.com/questions/30458195/does-gc-release-back-memory-to-os) for some more in-depth information. This one is possibly helpful for older java versions: https://stackoverflow.com/questions/675589/jvm-sending-back-memory-to-os — sorifiend, Jun 29 '22 at 02:09

score 0 · Answer 1 · answered Jun 29 '22 at 09:14

There are several places where excessive memory is used but everything within the method except the returned int[] is garbage collectable so you should not have any concerns.

However If you are reading in many values - say 100,000 or more then the suggestions below will reduce the memory footprint needed.

Presize ArrayList before use avoids re-allocations when it grows:

int initialSize = 100000;
ArrayList<Integer> innt = new ArrayList<>(initialSize);

Avoid StringBuffer per parsed Integer. As int has maximum length you can replace with final char[] doubus:

final char[] doubus = new char[Integer.toString(Integer.MAX_VALUE).length() + 2];
int dIndex = 0;
// ... Then append with:
    if (a != '|') {
        doubus[dIndex++] = a;
    } else {
        innt.add(Integer.valueOf(new String(doubus, 0, dIndex)));
        dIndex=0;
    }

Using Integer.valueOf with ArrayList<Integer> means you are auto-boxing many int values as Integer, only to extract as int at the end. Swap out the ArrayList for int[] and use Integer.parseInt so the result is always primitive type int means many memory conversions are avoided:

int [] innt = new int[initialSize];
int iIndex = 0;
// Replace add with:
    innt[iIndex++] = Integer.parseInt(new String(doubus, 0, dIndex));

Putting this together you should have same output and hopefully less memory churn:

public static int[] findDoub(File file) throws IOException {
    int initialSize = 100000; // Use suitable value
    int [] innt = new int[initialSize];
    int iIndex = 0;

    try (BufferedInputStream buff = new BufferedInputStream(new FileInputStream(file))) {
        int index = buff.read();

        final char[] doubus = new char[Integer.toString(Integer.MAX_VALUE).length() + 2];
        int dIndex = 0;

        for (int i = 0; index != -1; i++) {
            char a = (char) index;
            if (i > 0) {
                if (a != '|') {
                    doubus[dIndex++] = a;
                } else {
                    // Grow int[] if needed:
                    if (iIndex == innt.length) {
                        innt = Arrays.copyOf(innt, innt.length + initialSize);
                    }
                    innt[iIndex++] = Integer.parseInt(new String(doubus, 0, dIndex));
                    dIndex=0;
                }
            }
            index = buff.read();
        }
    }
    // Return exact size int[] of current length:
    return Arrays.copyOf(innt, iIndex);
}

score -1 · Answer 2 · answered Jun 29 '22 at 01:19

-1

you read all the file to innt until the function end. and you copy it to innnt. the

both causing ram usage.

ram usage also depends on you file size.

you dont need copy innt, just return arraylist, it will improve ram usage.

and you can try split the file.

answered Jun 29 '22 at 01:19

sheng

69
3

It uses ram even after the method returns – Juan_Puthr Jun 29 '22 at 02:03
your gc need call after last foreach and before return, them innt will be recycle – sheng Jun 29 '22 at 03:41
method returns but innnt still has been ues, so it should has some memory use. if you dont want any memory use, try change way to get result. like save innnt to another file, then gc and return. – sheng Jun 29 '22 at 03:48

My method is for some reason using up memory even after it finishes

2 Answers2