0

My program executes system commands, and returns the output line by line, however, there are a couple of commands that produces lots of lines, in this case the RAM usage rises to ~700Mbs, knowing that the usual RAM usage in any other commands is 50-60Mbs.

This is the method that handles reading the output using BufferedReader, it is called by another method that handles the creation of the process of the command. it also passes the output line by line to showOutputLine() method, which will print it to the console or to a TextArea. 

    protected void formatStream(InputStream inputStream, boolean isError) {

    bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
    String tempLine = null;

    // Read output
    try {
        while ((tempLine = bufferedReader.readLine()) != null) {
            showOutputLine(tempLine, isError);
        }
    } catch (IOException e) {// just stop
    }

}

one example of the commands that causes the issue:

adb logcat

EDIT: it appears BufferedReader is innocent, however, the problem still persists. caused by JTextArea.

mhashim6
  • 527
  • 6
  • 19
  • 1
    No it doesn't. The TextArea does. – user207421 May 01 '17 at 14:46
  • I tested that, I printed the output to the console, neglected the textArea entirely, but that didn't solve the problem. – mhashim6 May 01 '17 at 14:48
  • Can you share the implementation of showOutputLine? – yonisha May 01 '17 at 14:51
  • line 124: https://github.com/mhashim6/Commander/blob/master/src/mhashem6/commander/ExecuterStreamFormatter.java outputObject is an instance of an interface, that the user can configure to redirect the output whether to the console or any other preferred object. – mhashim6 May 01 '17 at 14:54
  • 1
    Try not printing the output at all, or not calling anything inside the read loop.. `BufferedReader` only uses a few K. It is not the problem here. – user207421 May 01 '17 at 15:19
  • this worked fine, the RAM usage was ~40Mbs. – mhashim6 May 01 '17 at 15:30
  • QED. So the problem is elsewhere. – user207421 May 01 '17 at 15:35
  • if it was the UI implementation, is there any way to handle such huge output properly? knowing that I really need to pass the output line by line, not to wait until the process finishes. – mhashim6 May 01 '17 at 15:41
  • Err, don't try to display an arbitrary amount of text in the UI? If it gets even moderately large you'll have to provide scrolling facilities, and at some larger point you'll have to provide search facilities. Somewhere in there you should be looking at paging the UI display from a file. – user207421 May 02 '17 at 09:51
  • I'm already implementing scrolling, I've also thought about redirecting the output to a file first, but that will not work with a command like logcat, because it doesn't exit, it keeps logging to the stream, however, I've thought about a work around: instead of displaying a line by line (String object by string object) I will make a String for each (say 4 lines), that did actually reduce the usage to 160MBs, I can enhance it, but it still is a work around, and it's painful to implement in my program, because there are some other commands that produce just one line of output. – mhashim6 May 02 '17 at 10:27
  • Now I removed the TextArea again, rerun, and no problems at all, my bad, however, is this a bug in the TextArea or there's s solution to such huge output situations? – mhashim6 May 02 '17 at 13:27
  • It's a design bug in your code. You can't put large amounts of text anywhere, let alone a `TextArea`, without consuming memory. – user207421 May 02 '17 at 23:01
  • Why? The console can handle it, Eclipse's console did handle it, and I monitored eclipse's consumption, and it didn't rise that dramatically. – mhashim6 May 03 '17 at 07:32

1 Answers1

1

BufferedReader always uses about 16 KB (8K * 2 byte chars) in a fixed size array. If you are using more than this it is a side effect of generating so many Strings (esp if you have really long lines of text) not the BufferedReader itself.

TextArea can retain much more memory usage depending on how long the text is.

In any case, the memory usage which really matters is the size of the heap after a Full GC, the rest is overhead of various kinds.

BTW Mb = Megi-bit, MB = Mega-byte.

Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130
  • So any suggestions about handling this properly with UI? Please read my comments on the OP if possible, to have an idea of my particular situation. – mhashim6 May 02 '17 at 13:17
  • @mh6 keep the amount of data in the JTextArea to a minimum or recognise this will use more memory. In particular it is likely to buffer an image around the whole body of text as a rectangle. Large amounts of text (esp if there is a long line) use a lot of memory. – Peter Lawrey May 03 '17 at 07:33