JRuby PermGen. Out of memory. Heap space memory leaks

Question

We are working on a big project. During our development we faced with a big a problem related to Heap and PermGen Space. Here our error message: Current error: java.lang.OutOfMemoryError: PermGen space.

System.out.println("Initialazing..");
//Spring applicaton context
WebApplicationContext wac = (WebApplicationContext) AppContext.getApplicationContext();
// prepare path to internal ruby 
String scriptsPath = wac.getServletContext().getRealPath(RUBY_PATH);
String jrubyHome = wac.getServletContext().getRealPath("WEB-INF" + File.separator + "jruby");
// Initializing Scripting container
ScriptingContainer container = new ScriptingContainer(isShared ? LocalContextScope.SINGLETHREAD
                : LocalContextScope.THREADSAFE, LocalVariableBehavior.PERSISTENT);
// Configuring scriptingcontainer to avoid memory leaks
container.setCompileMode(RubyInstanceConfig.CompileMode.OFF);
System.setProperty("org.jruby.embed.compilemode", "OFF");
System.setProperty("jruby.compile.mode", "OFF");
// Setup ruby version
container.setCompatVersion(CompatVersion.RUBY1_9);
// Set jruby home
container.getProvider().getRubyInstanceConfig().setJRubyHome(jrubyHome);
List<String> loadPaths = new ArrayList<String>();
// load path
loadPaths.add(scriptsPath);
container.getProvider().setLoadPaths(loadPaths);
// ruby dispatcher initializing and run in simple mood
String fileName = scriptsPath + File.separator + "dispatcher_fake.rb";
// run scriplet
container.runScriptlet(PathType.ABSOLUTE, fileName);
// terminate container to cleanup memory without any effects
container.terminate();
container=null;

…

Description: above code create and configure Scripting container. This methods running in separate thread. We use 4 threads of ruby runs. If we use the same scripting container and call internal scriptlets (call internall methods in java threading) we get issues with ruby variables because it is visible cross threading.

Main issue with JRuby are: growing heap memory space and perm gen memory space. We can't call any system's garbage collection in the ruby code.

Bellow you can found simple parts of our scriptlet: Ruby:

ENV['GEM_PATH'] = File.expand_path('../../jruby/1.9', __FILE__)
ENV['GEM_HOME'] = File.expand_path('../../jruby/1.9', __FILE__)
ENV['BUNDLE_BIN_PATH'] = File.expand_path('../../jruby/1.9/gems/bundler-1.0.18/bin/bundle', __FILE__)

require 'java'
require 'rubygems'
require "bundler/setup"
require 'yaml'
require 'mechanize'
require 'spreadsheet'
require 'json'
require 'rest-client'
require 'active_support/all'
require 'awesome_print'
require 'csv'
require 'builder'
require 'soap/wsdlDriver' rescue nil
ROOT_DIR = File.dirname(__FILE__)
require File.join(ROOT_DIR, "base", "xsi.rb")


import org.jsoup.Jsoup
import org.jsoup.nodes.Document
import org.jsoup.nodes.Element


module JavaListing
  include_package "com.util.listing"
end

class A
  include JavaListing
  def run
    1000.times do |index|
      puts "iterating #{index}"
      prop = JavaListing::Property.new
      prop.proNo = 111
      prop.remoteID = "1111"
      prop.ownerID = "1111"
      prop.advertiserID = "1111"
      prop.title = "Atite"
      prop.summary = "Asummury"
      prop.description = "Adescription"
      # prop.images << JavaListing::Image.new("111", "Acaption")
      prop.lat = 12.23
      prop.lng = 13.21
      #prop.address = JavaListing::Address.new("Acity", "Acountry")
      prop.location = "Alocation"
      prop.policy = JavaListing::Policy.new("AcheckinAt", "AcheckoutAt")
      prop.surfaceArea = "Asurfscearea"
      prop.notes[index] = JavaListing::Note.new("Atitle", "Atext")
      prop.order = "Aorder"
      prop.map = JavaListing::Map.new(true, 14)
      prop.units[index] = JavaListing::Unit.new("Aproptype", 2)
      obj = Nokogiri::XML "<root><elements><element>Application Error  #{index}          </element></elements></root>"

    end
  end
end

A.new.run

The same perm gen with other kinds of scriptlets containers:

Create properties using JSR223

ScripHelperBase.java

ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("jruby");
Reader reader = null;
String fileName = scriptsPath + File.separator + "dispatcher_java.rb";
try {
    reader = new FileReader(fileName);
} catch (FileNotFoundException ex) {
    Logger.getLogger(ScriptHelperBase.class.getName()).log(Level.SEVERE, null, ex);
}
engine.eval(reader);

Ways to solution

1. List item

2. Increase PermGen space PermGen space was increased to 4Gb. Soon it became full too.

Create properties using BSF

String fileName = scriptsPath + File.separator + "dispatcher_fake_ruby.rb";
String jrubyhome = "WEB-INF" + File.separator + "jruby";

BSFManager.registerScriptingEngine("jruby", "org.jruby.embed.bsf.JRubyEngine", new String[]{"rb"});
BSFManager manager = new BSFManager();
manager.setClassPath(jrubyhome);

try {
    manager.exec("jruby", fileName, 0, 0, PathType.ABSOLUTE);
} catch (BSFException ex) {
    Logger.getLogger(ScriptHelperBase.class.getName()).log(Level.SEVERE, null, ex);
}

Conclusion: This mean that just increasing available space isn’t a solution of this hot problem.

The method described above don’t allow to clear needed memory to original state. It means that every additional script running still increase filled PermGen space.

Running system using CompileMode=OFF

container.setCompileMode(RubyInstanceConfig.CompileMode.OFF);
System.setProperty("org.jruby.embed.compilemode", "OFF");
System.setProperty("jruby.compile.mode", "OFF");

Answer 1

JRuby creates Ruby Proxies for native Java classes you are using - they are new classes created at runtime and by default JVM keeps them in memory for ever.

Here is solution.