Garbage Collection, Tomcat, Hibernate, and You

java.lang.OutOfMemoryError: PermGen space

If you are using Sun's Hotspot JVM, maybe you've seen this in your log file, and you don't know what to do. Maybe this will help.

Ever since we upgraded at work from Hibernate 2.1 to 3.2, we have been fighting memory management issues. Some of those can probably be traced back to increased traffic, some probably to odd legacy code that can be refactored pretty easily, and much of it relates to Hibernate and cgLib. There are endless blog posts about this, so if you want to read about the way cgLib, Hibernate, and the Perm region of the JVM heap interact, google away.

To break it down, here's how to know you may run across a problem:

/bin/jps

This will give you a listing of running JVM instances on your machine. Tomcat presents as:

xxxx Bootstrap

this is the JVM pid that you can use to check out memory utilization. Now run the garbage collection util to see what's going on in your heap:

jstat -gcutil

You will be presented with the following:

S0	S1	E	O	P	YGC	YGCT	FGC	FGCT	GCT
0.00	31.58	60.13	57.76	99.66	2647	5.192	6	2.114	7.306

The measures above give you space utilization in percentage of the regions of the JVM memory space. For a detailed explanation of this, check out the official Sun paper on the topic. For our purposes, we will do a quick overview:

S0 and S1 are both survivor spaces.
E is Eden space.
O is Old space.
P is Perm space.

When an object is created, it lives in Eden space. If it makes it past a garbage collection while still containing an active reference to it, it will move on to Survivor space. From there, if it's still actively referenced, it moves on to Old space. Garbage Collections of Eden and Survivor spaces are not the big garbage collections - the are the Young Garbage Collections (YGC). Once something gets into the Old space, it can only be removed by a Full Garbage Collection (FGC). You can see from the above table that a YGC is not expensive - 2647 of them were performed in a total of 5.192 seconds, while just six full collections take 2.114 seconds. Finally, there is the Perm space. This is where the JVM structures and class objects are put. The classloader sticks stuff here to help you so it doesn't have to constantly load and reload these structures. The problem is that if you have a lot of these things, it's going to just keep sticking things in there until it's full.

There are a couple of ways to combat this issue:

1. Maybe you haven't increased the default size of the Perm space. You can do that by adding this JVM flag to your startup script: -XX:MaxPermSize=m. To give some perspective, the default is 32m.
   
2. Look at your classpath. Are you loading a LOT of libraries in? Do you need them all? Perhaps you can remove some of those libraries.

If you still have problems with Perm space after making the above changes, try these JVM arguments and see if you have any more issues.

-XX:+UseConcMarkSweepGC -XX:+CMSPermGenSweepingEnabled -XX:+CMSClassUnloadingEnabled

Whereas we were constantly hovering around 99.88% perm space utilization prior to these changes, now we are usually back down around 60%, so an extraordinary event in the system won't trigger an outofmemory situation. Hope this helps.