When to Turn On Verbose GC
From Zimbra :: Wiki
The Zimbra server is running slowly or is not responding to requests.
When major garbage collection is running, server activity pauses. If you notice that the server is not responding or is running slow, garbage collection might be running too frequently.
How to troubleshoot the problem
Run –verbose GC to review the frequency of collections. When you run the verbose GC command, make sure that you include the print timestamp option (-XX:+PrintGCTimeStamp) and the print details option (-XX:+PrintGCDetails). For troubleshooting GC issues, the timestamp and the frequency of garbage collection are important clues.
In a normal garbage collection, the full GC should take a few seconds to run depending on the total heap size. A YoungGen GC should take a few milliseconds. If you notice that the full GC collection occurs frequently, for instance it is running two or more times every five minutes, you may have to tweak the JVM configuration.
In the GC examples below the first detail in each record is the timestamp in seconds that shows when this record started. The timestamp is an offset of the server’s start time. In the first example, the timestamp 1750.263 shows that the server was started about 29 minutes before. The last detail in the record is the time the GC process took, in seconds. The Full GC took about 2 seconds.
Full GC collection – Time 2.0045200 seconds
1750.263: [Full GC[Unloading class sun.reflect.GeneratedConstructorAccessor24]… [PSYoungGen: 4400K->0K(499392K)] [PSOldGen: 900670K->805968K(901120K)] 905070K->805968K(1400512K) [PSPermGen: 30241K->30241K(60544K)], 2.0045200 secs]
Minor GC collection – 0.0244970 seconds
1762.580: [GC [PSYoungGen: 486784K->7667K(499648K)] 1292752K->813636K(1400768K), 0.0244970 secs]
Example of GC running too frequently
In the following example, - verbose:gc shows that a full GC is running about every 3 seconds. With Full GC running this often, the server is constantly in a pause state.
581.146: [Full GC [PSYoungGen: 170752K->0K(341376K)] [PSOldGen: 786611K->786713K(919552K)] 957363K->786713K(1260928K) [PSPermGen: 19076K->19076K(29184K)], 3.4611450 secs] 584.691: [Full GC [PSYoungGen: 170752K->0K(341376K)] [PSOldGen: 786713K->786759K(919552K)] 957465K->786759K(1260928K) [PSPermGen: 19076K->19076K(27648K)], 3.4172340 secs] 588.202: [Full GC [PSYoungGen: 170752K->0K(341376K)] [PSOldGen: 786759K->786047K(919552K)] 957511K->786047K(1260928K) [PSPermGen: 19137K->19137K(26624K)], 4.2417390 secs] 592.533: [Full GC [PSYoungGen: 170752K->0K(341376K)] [PSOldGen: 786047K->786218K(919552K)] 956799K->786218K(1260928K) [PSPermGen: 19147K->19147K(25600K)], 3.3467470 secs] 596.085: [Full GC [PSYoungGen: 170752K->0K(341376K)] [PSOldGen: 786218K->786421K(919552K)] 956970K->786421K(1260928K) [PSPermGen: 19158K->19158K(24576K)], 3.3797990 secs]
GC may be running too frequently because of the following type of issues:
• The heap size is too small. In ZCS, the heap size is a percentage of the RAM. To see the Java heap memory percentage for Tomcat, type:
zmlocalconfig –e tomcat_java_heap_memory_percent.
• The server configuration is causing GC to run too often. For instance, if the indexing cache size is set too large and high loads are exercising the index, the server will need more memory. Review your server mailbox configuration settings, CLI command zmlocalconfig.
• Server memory leak – If you suspect that you have a server memory leak, contact Zimbra Support.
Changing Young Generation Size
The bigger the young generation, the less often minor garbage collection occurs. But a larger young generation implies a smaller old generation, which may increase the frequency of full GC. The NewSize and MaxNewSize parameters determine the new generation’s minimum and maximum size.
In most cases you should not change the default settings. But, if the server is fully loaded and you are seeing frequent full GC Runs, decreasing the NewSize will cause more frequent minor GC runs. This will decrease the frequency of full GC runs.
Caution: If you set the NewSize too small, the survived live objects will be promoted to tenured generation and a full GC will be forced to happen.
Note: If your server’s load is close to the limit, tuning NewSize may cause big performance differences.
|Verified Against:||Date Created: 11/5/2007|
|Article ID: http://wiki.zimbra.com/index.php?title=When_to_Turn_On_Verbose_GC||Date Modified: 03/17/2010|