5

Glassfish sometimes fails to stop when the PermGen is full, in this case asadmin stop-domain domain1 doesn't work. In Glassfish 2.1.1 it would just sit there forever; in 3.x it times out after AS_ADMIN_READTIMEOUT. So I am now working on my Glassfish stop script that will kill/kill -9 it after a certain timeout to guarantee it gets stopped.

To fully test this I need to reproduce this PermGen full scenario. How can I deliberately fill up PermGen? I'm currently using Java 1.7.0_45 if that matters. I've written a program to fill up heap but this is a new one for me, I figured I'd turn to SO. It might be trickier (not sure) that it needs to be something (a .war?) I can deploy into GF. Any help is appreciated, thanks so much.

Steve Kehlet
  • 6,156
  • 5
  • 39
  • 40
  • 1
    you'll have to create a lot of ClassLoaders. I never tried that, but you should look that way. – EasterBunnyBugSmasher May 01 '14 at 19:56
  • What do you know about `PermGen`? How do objects end up there? Once you answer these questions you will be able to work out easily how to fill `PermGen` for your testing. – Germann Arlington May 01 '14 at 20:31
  • @EasterBunnyBugSmasher how would multiple ClassLoaders help? – Germann Arlington May 01 '14 at 20:31
  • @GermannArlington yeah the comment from EasterBunnyBugSmasher has me investigating a custom ClassLoader that will fabricate any class I ask it for, and then I'll have to ask it for a scrillion different classes in a loop. – Steve Kehlet May 01 '14 at 20:33
  • You don't meed a lot of different classes to fill `PermGen`, just a lot of long living objects... even of one class – Germann Arlington May 01 '14 at 20:35
  • 1
    well yes, you don't need a lot of ClassLoaders. One will be enough. How about this: you make a Class file template that you can copy and on each occasion change the class name in it and the file name. The file must be in your classpath. Then you do ClassLoader.getSystemClassLoader().loadClass("MyClass" + i); – EasterBunnyBugSmasher May 01 '14 at 20:41
  • Couldn't you follow this example: http://stackoverflow.com/questions/5178391/create-simple-pojo-classes-bytecode-at-runtime-dynamically and just make a metric ton of dynamic classes? Meaning use ASM or the CGLib api's? Also, no permagen in java 8 :D – Mark W May 01 '14 at 21:07
  • See http://stackoverflow.com/questions/9802626/programmatically-fill-the-jvm-permanent-generation-permgen-memory-region. A solution mentioned is to retain a lot of references to different interned strings, as these are added to the string pool (which is located in perm gen in java 6, seems this isn't the case for later versions see http://java-performance.info/string-intern-in-java-6-7-8/ which suggests the string pool was moved). – GoldenJam May 01 '14 at 21:15
  • @EasterBunnyBugSmasher thank you, your solution of creating a ton of classes and then just loadClass()ing them was very simple and worked great. If you make it an answer I can accept it. – Steve Kehlet May 01 '14 at 22:16
  • @MarkW I tried the way you suggested (great idea), unfortunately I don't understand why but the PermGen didn't seem to grow, even though I was instantiating new classes through CGLIb in a loop. So I gave up :-(. – Steve Kehlet May 01 '14 at 22:17
  • @GoldenJam nice find, my question is a total duplicate. Unfortunately as you pointed out String handling is different in Java 7 now, so I couldn't use that answer. – Steve Kehlet May 01 '14 at 22:18
  • 1
    @Steve Kehlet its a little late now I think, but the reason is probably because you must make meaningful use of the class before its put on the permagen, or specifically load the class via the class loader. I dont know if CGLib will load the classes immediately or not, but if I had to guess, that would have been it. – Mark W May 02 '14 at 13:38

1 Answers1

6

I have some thing for you. I don't know how to upload a jar file here so just adding files here.

Approach: ClassGenerator class creates a new class loader in a while loop and loads the same class over and over again until it run out of permgen. Now you will notice there is a list that keeps a reference of the loaded class. That is to prevent JVM from unloading those classes :).

Files Explained
The first image shows that when you run the program it run out of permgen space. Second image shows the structure of the project if you want to set it up in eclipse. I tested it within eclipse and exported it as a runnable jar file it works in both cases.

Run as a runnable jar file and it ran out of permgen. Program ran out of permgen

Eclipse Project setup

enter image description here

ClassGenerator class

package com.vkg;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;

public class ClassGenerator {
    private static final int BUFFER = 1024;
    private List<Class<?>> classList = new ArrayList<Class<?>>();

    public static void main(String[] args) {
        ClassGenerator classGenerator = new ClassGenerator();
        // Load just some class with class loaders until perm gen space fills.
        while (true) {
            classGenerator.classLoader();
        }
    }

    private void classLoader() {
        ClassLoader classLoader = new ClassLoader() {
            public Class<?> loadClass(String classNameWithPackage)
                    throws ClassNotFoundException {
                if (!classNameWithPackage.contains("DummyClass")) {
                    return super.loadClass(classNameWithPackage);
                }
                String className = classNameWithPackage.replace('.', '/')
                        + ".class";
                byte[] classData = null;
                InputStream inputStream = null;
                try {
                    inputStream = getResourceAsStream(className);
                    byte[] buffer = new byte[BUFFER];
                    ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
                    int bytesRead = -1;
                    while ((bytesRead = inputStream.read(buffer, 0, BUFFER)) != -1) {
                        outputStream.write(buffer, 0, bytesRead);
                    }
                    classData = outputStream.toByteArray();
                } catch (IOException e) {
                    e.printStackTrace();
                } finally {
                    if (inputStream != null) {
                        try {
                            inputStream.close();
                        } catch (Exception e) {
                            e.printStackTrace();
                        }
                    }
                }
                Class<?> c = defineClass(classNameWithPackage, classData, 0,
                        classData.length);
                resolveClass(c);
                System.out
                        .println("Steve I am loading another copy of Dummy class. Soon Permgen will fill.");
                classList.add(c);
                return c;
            }
        };

        try {
            Class.forName("com.vkg.DummyClass", true, classLoader);
        } catch (ClassNotFoundException e) {
            e.printStackTrace();
        }
    }
}

Dummy class. This can be any class. Sole purpose of this class is to get loaded large number of times. No other use. No logic gets executed from this class. Main logic is in ClassGenerator.java

package com.vkg;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;

public class DummyClass {
    public void classLoader() {
        ClassLoader classLoader = new ClassLoader() {
            public Class<?> loadClass(String classNameWithPackage) throws ClassNotFoundException {
                 if(!classNameWithPackage.contains("DummyClass")) {
                     return  super.loadClass(classNameWithPackage);
                 } 
                 String className = classNameWithPackage.replace('.', '/') + ".class";
                byte[] classData = null;
                try {
                    InputStream inputStream = getResourceAsStream(className);
                    byte[] buffer = new byte[1];
                    ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
                    int bytesRead = -1;
                    while ((bytesRead = inputStream.read(buffer, 0, 1)) != -1) {
                        outputStream.write(buffer, 0, bytesRead);
                    }
                    classData = outputStream.toByteArray();
                }
                catch (IOException e) {
                    e.printStackTrace();
                }

                Class<?> c = defineClass(classNameWithPackage, classData, 0, classData.length);
                resolveClass(c);
                System.out.println("Steve I am loading another copy of Dummy class. Soon Permgen will fill.");
                return c;
            }
        };

        try {
            Class.forName("com.vkg.DummyClass", true, classLoader);
        } catch (ClassNotFoundException e) {
            e.printStackTrace();
        }
    }
}

Hope it helps you test your server crash.

rene
  • 41,474
  • 78
  • 114
  • 152
vkg
  • 1,839
  • 14
  • 15
  • I'm trying it out! Trying to figure out, why is there duplicate code in DummyClass and ClassGenerator? – Steve Kehlet May 01 '14 at 22:29
  • As I mentioned in the comment. It doesn't matter you can have any thing in the DummyClass. Sole purpose of that class is to get loaded n number of times. Really no logic gets executed from that class. – vkg May 01 '14 at 22:31
  • Interesting, I'm trying to get this to work and discovered (via `lsof`) it's opening thousands of file descriptors, and so I'm getting an NPE instead of the expected PermGen full. Working on it... – Steve Kehlet May 01 '14 at 22:41
  • Add a finally {} block to close that inputStream and it works great! See [my gist](https://gist.github.com/skehlet/a1d12bf81f8617a9593e). Please update your answer and I'll accept. Thanks so much. – Steve Kehlet May 01 '14 at 22:52
  • 1
    Thanks Steve. I agree with your proposed changes. I have incorporated them.:) – vkg May 01 '14 at 23:22