Interpreting jstat results

Question

I am new to jstat tool. Therefore I did a sample as below.

./jstat -gcutil -t 4001 5000
Timestamp         S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT
       565088.4   0.00   0.89  75.86  40.59  84.80    405    3.822     4    0.549    4.371
       565093.4   0.00   0.89  77.81  40.59  84.80    405    3.822     4    0.549    4.371
       565098.4   0.00   0.89  77.81  40.59  84.80    405    3.822     4    0.549    4.371
       565103.5   0.00   0.89  77.85  40.59  84.80    405    3.822     4    0.549    4.371
       565108.5   0.00   0.89  77.85  40.59  84.80    405    3.822     4    0.549    4.371
       565113.4   0.00   0.89  77.85  40.59  84.80    405    3.822     4    0.549    4.371


jstat -gc output

 S0C    S1C    S0U    S1U      EC       EU        OC         OU       PC     PU    YGC     YGCT    FGC    FGCT     GCT
704.0  704.0   0.4    0.0    6080.0   4013.8   14928.0     6335.2   21248.0 18019.6    436    3.957   4      0.549    4.506
704.0  704.0   0.4    0.0    6080.0   4016.6   14928.0     6335.2   21248.0 18019.6    436    3.957   4      0.549    4.506
704.0  704.0   0.4    0.0    6080.0   4135.4   14928.0     6335.2   21248.0 18019.6    436    3.957   4      0.549    4.506
704.0  704.0   0.4    0.0    6080.0   4135.4   14928.0     6335.2   21248.0 18019.6    436    3.957   4      0.549    4.506
704.0  704.0   0.4    0.0    6080.0   4135.4   14928.0     6335.2   21248.0 18019.6    436    3.957   4      0.549    4.506
704.0  704.0   0.4    0.0    6080.0   4135.4   14928.0     6335.2   21248.0 18019.6    436    3.957   4      0.549    4.506

What does this results indicates? Which are the columns to look out for possible memory problem e.g. memory leak etc.

score 32 · Accepted Answer · edited Nov 15 '19 at 15:44

32

See the documentation:

https://docs.oracle.com/javase/8/docs/technotes/tools/unix/jstat.html

Basically one row is one point in time. The columns show data about the JVM memory areas (Survivor, Eden, ...), understanding them is impossible without knowing how the JVM works.

For example in the article JVM garbage collection in young generation there is some explanation.

Here is the excerpt how JVM object generation works:

Eden is a place where new objects created. When the Eden is full, a small GC is run: if an object has no reference to it, it will be deleted, otherwise it will survive, and move to the Survivor space (only one of the survivor spaces in use at a time, all objects from the other space is copied there).

If an object survives a certain number of back-and-forth copying, it is moved to Old space. If the Old space is full, a Full GC is run, which affects all objects in the JVM, so it is much heavier operation.

Also, there is the Permanent space, where the "metadata" (class descriptors, field, method, ... descriptors) are stored.

edited Nov 15 '19 at 15:44

Eugene

117,005
15
201
306

answered Jan 22 '13 at 18:00

gaborsch

15,408
6
37
48

I have google too but I dont quite get this part S0C Current survivor space 0 capacity (KB). S1C Current survivor space 1 capacity (KB). as most of the sites discuss regarding this. – new14 Jan 22 '13 at 18:07
This is not the size, it's the capacity (in %). Have a look at this: http://stackoverflow.com/questions/13660871/jvm-garbage-collection-in-young-generation/13661014#13661014 There is som explanation about the object generation. – gaborsch Jan 22 '13 at 18:12
so in my case the eden space is 77.85 which soon will be having a minor GC is it? So at present the eden is now nearly 94% what does this indicate when the eden grow meaning new stuff is added is it? – new14 Jan 22 '13 at 18:29
Does it mean that my old stufed are not cleared automatically and can this lead to memory leakage? – new14 Jan 22 '13 at 18:30
Eden reach 100 then the S0 becomes 0.89 and S1 is 0.00 and then E start over and S0 is 0.00 and S1 back to 0.89? – new14 Jan 22 '13 at 18:33
You don't have to worry, GC runs quite frequently. See update for some explanation. – gaborsch Jan 22 '13 at 23:38
In your case I would add more `PermGen` space, 84% is a bit high. And yes, if PermGen is full, it can cause `OutOfMemoryError`. – gaborsch Jan 22 '13 at 23:58
now I notice is 80.87% there is a drop. How will it get a drop? How much PermGen to be added? – new14 Jan 23 '13 at 04:35
THe PermGen itself also has a minimum size, actual size, and maximum size. Probably when it was 84%, the size was smaller. Later it has been extended, do the percent ws dropped. – gaborsch Jan 23 '13 at 09:35
I have accepted it but I want to learn and confirm more things from you. How do I do the percent vs dropped? What else should be on the looked out? – new14 Jan 23 '13 at 10:42
When I run gccause it shows me "Allocation Failure No GC". What does this means Allocation Failure? – new14 Jan 23 '13 at 10:45
If you run `jstat -gc` you will get the capacities and the utilisation percentage as well. There you can see the numbers. I cannot give you a course on jstat :) but I can tell what I would see: `Old` and `PermGen` size, the frequency of the small GC (it tells about the object usage, if it's frequent I'd add more to Eden). Also, if it's a critical app, I would check it more frequently and compared the result (as users use the app, how the memory spaces are growing) – gaborsch Jan 23 '13 at 10:49
On Allocation Failure No GC" : look at http://www.ibm.com/developerworks/library/i-gctroub/ and see "Avoiding heap trashing" - there is a solution there. – gaborsch Jan 23 '13 at 10:55
I updated my question with jstat -gc output so what should I look is the allocated and utilisation ? Yes I understand you cant give me a course hehe but just highlight to me what should be on the look out as my app is a critical one. – new14 Jan 23 '13 at 13:16
I have visited the link you gave so that means I must increate a bigger heap size to avoid this problem is it? Actually my application is quite small but why is it having allocation failure – new14 Jan 23 '13 at 13:21
Maybe because the GC always frees some memory, but it's not enough for your app's needs. E.g. you want to allocate a large array (e.g. a `String` has a `char[]` inside), and there is not enough memory for that. You cannot say that the memory is full, it's just not enough. – gaborsch Jan 23 '13 at 13:34
Yes I use quite a number of places String variables. Another thing I have also few lines to monitor the memory in the application itself. System.out.println("Used Memory:" + (runtime.totalMemory() - runtime.freeMemory()) / mb); I notice there is some small increase like one or 2 mb after sometime. – new14 Jan 23 '13 at 14:18
It is interesting if you want to store a String with 1M (e.g. even with an implicit `stringBuilder.toString()`). If the GC runs, it will free up some memory, probably that's what you can see. 2M is quite small. From the latest stats I can see that you should increase the `PermGen` space, because that is almost full. – gaborsch Jan 23 '13 at 14:33
I dont get you when you say "if you want to store a String with 1M". How much should I add the PermGen? So during GC wouldnt the PermGen get clearer. – new14 Jan 23 '13 at 16:00
These are 2 different stories. Actually, both the `-Xmx` and the `-XX:MaxPermSize` parameters should be raised. – gaborsch Jan 23 '13 at 16:07
so for a 4Gb machine what are the best parameters set for both -Xmx and -XX:MaxPermSize? – new14 Jan 23 '13 at 16:14
It depends. What are the current settings? Is it desktop or server app? What OS you have? What other apps are running on it? What user load is expected (compared to the current load)? Generally, you can give 3G to -Xmx and 512M - 768M to permsize as a maximum. – gaborsch Jan 23 '13 at 16:38
Please consider using a room on [chat]; it's really much easier to hold a conversation there. – Michael Myers Jan 23 '13 at 16:42
@MichaelMyers you're right, but unfortunately new14 has not enough reputation to use it – gaborsch Jan 23 '13 at 16:45
I am using Centos 6.3. I am not sure how to check out the current settings. The app is socket listener where it is a server and clients will connect and pass data to it. The cients here devices which will connect and pass data. – new14 Jan 23 '13 at 16:46
Check the script where you start the app, probably there are the current options. If you find it, you can modify them. If not, you use your JVM's default settings (which is usually about 256M), in this case add the params you want and start playing around. Try 1G / 128M for start. – gaborsch Jan 23 '13 at 16:58
Ok I will try out with Xmx as 1G and MaxPermSize as 128M. Another thing System.out.println("Max Memory:" + runtime.maxMemory() / mb); gives the values as 1735? IS this the Xmx value. – new14 Jan 23 '13 at 17:09
@GaborSch I notice there is an increase of 1Mb over some few hours. What else tools can I use to confirm if I have some memory issues? – new14 Jan 24 '13 at 17:45
1MB is not much. In fact, that's very low. I don't know that libraries you use, but imagine that every bit of code you use in this JVM can add something (classes, objects, caches, factories, sessions, pools, whatever construct they use) to the allocated memory. So if you have 1G, calculate that when would it reach the maximum available memory - weeks. But that's another time scale, you will see that the rate of the increase is dropping - it's kind of warm-up. – gaborsch Jan 24 '13 at 17:54
I also recommend to look around that tools you find in the `$JAVA_HOME/bin`. I recommend `jmap` - http://docs.oracle.com/javase/1.5.0/docs/tooldocs/share/jmap.html – gaborsch Jan 24 '13 at 17:59
@GaborSch ok basically my application is socket listener and there after it have a number select,insert and update statement are run. In addition I am also using bonecp for connection pooling. I have tried using the jmap and manage to take the output. So what are key things I should look out in the jmap. I will post a sample of it? – new14 Jan 25 '13 at 14:00
Yes, I'd recommend to post a new question about that. But before posting a question, please look around if there are similar (already answered) questions there, otherwise you will be downvoted. – gaborsch Jan 25 '13 at 14:04
@GaborSch actually I have managed to build snapshots using this method ./jmap -dump:format=b,file=snapshot2.bin 12671 and view it via visual vm tool and compare snapshot is that good enough to indicate? – new14 Jan 25 '13 at 17:03
Look for oddities. For example there are too many instances of (YourClass). – gaborsch Jan 25 '13 at 17:19
I saw this java.lang.outofMemoryError 8 instance but the bigger instances are of of char =32367 and java.lang.String =28761. So what does this number indicte that this many characacter have been use or in use ? – new14 Jan 26 '13 at 17:27

score 32 · Answer 2 · edited Jan 19 '15 at 12:39

gcutil gives stats in terms of percentage utilization

-gcutil Option
Summary of Garbage Collection Statistics 
Column  Description
S0      Survivor space 0 utilization as a percentage of the space's current capacity.
S1      Survivor space 1 utilization as a percentage of the space's current capacity.
E       Eden space utilization as a percentage of the space's current capacity.
O       Old space utilization as a percentage of the space's current capacity.
P       Permanent space utilization as a percentage of the space's current capacity.
YGC     Number of young generation GC events.
YGCT    Young generation garbage collection time.
FGC     Number of full GC events.
FGCT    Full garbage collection time.
GCT     Total garbage collection time.

gc gives statistics in terms of alloted space and utilized space.

-gc Option
Garbage-collected heap statistics 
Column  Description
S0C     Current survivor space 0 capacity (KB).
S1C     Current survivor space 1 capacity (KB).
S0U     Survivor space 0 utilization (KB).
S1U     Survivor space 1 utilization (KB).
EC      Current eden space capacity (KB).
EU      Eden space utilization (KB).
OC      Current old space capacity (KB).
OU      Old space utilization (KB).
PC      Current permanent space capacity (KB).
PU      Permanent space utilization (KB).
YGC     Number of young generation GC Events.
YGCT    Young generation garbage collection time.
FGC     Number of full GC events.
FGCT    Full garbage collection time.
GCT     Total garbage collection time.

Source : Docs

danixon · Answer 3 · 2016-08-18T03:45:08.003

20

Use this simple online jstat visualizer tool to plot jstat GC statistics.

edited Aug 18 '16 at 03:45

answered Jan 02 '15 at 22:16

danixon

301
2
5

7

Rather than post links (which can die), you should provide the necessary information in your answer. – bencripps Jan 02 '15 at 22:26

Interpreting jstat results

3 Answers3

Linked