3

In a Java servlet container (preferably Tomcat, but if this can be done in a different container then say so) I desire something which is theoretically possible. My question here is whether tools exist to support it, and if so what tools (or what names I should research further).

Here is my problem: in one servlet container I want to run a large number of different WAR files. They share some large common libraries (such as Spring). At first blush, I have two unacceptable alternatives:

  1. Include the large library (Spring, for example) in each WAR file. This is unacceptable because it will load a large number of copies of Spring, exhausting the memory on the server.

  2. Place the large library in the container classpath. Now all of the WAR files share one instance of the library (good). But this is unacceptable because I cannot upgrade the Spring version without upgrading ALL of the WAR files at once, and such a large change is difficult verging on impossible.

In theory, though, there is an alternative which could work:

  1. Put each version of the large library into the container-level classpath. Do some container level magic so that each WAR file declares which version it wishes to use and it will find that on its classpath.

The "magic" must be done at the container level (I think) because this can only be achieved by loading each version of the library with a different classloader, then adjusting what classloaders are visible to each WAR file.

So, have you ever heard of doing this? If so, how? Or tell me what it is called so I can research further.

Raedwald
  • 46,613
  • 43
  • 151
  • 237
mcherm
  • 23,999
  • 10
  • 44
  • 50
  • No answers yet. Is there something I can do with OSGi that will achieve this? Write my own classloader that somehow cooperates with the complex classloader hierarchy that Tomcat (or whatever container I use) is creating? – mcherm Sep 23 '14 at 13:11
  • OSGi supports this out-of-the-box. Also putting Spring jar (or other library jars in the container for that matter) can lead to suprising results at times, sometimes frameworks use singletons inside (the wrong kind) and those are singleton per classloader not application! So you can unwillingly (and maybe unknowingly) share state between applications. – M. Deinum Sep 23 '14 at 13:25

3 Answers3

5

Regarding Tomcat, for the 7th version you can use VirtualWebappLocader like so

<Context>
    <Loader className="org.apache.catalina.loader.VirtualWebappLoader"
            virtualClasspath="/usr/shared/lib/spring-3/*.jar,/usr/shared/classes" />
</Context>

For the 8th version Pre- & Post- Resources should be used instead

<Context>
    <Resources>
        <PostResources className="org.apache.catalina.webresources.DirResourceSet"
                       base="/usr/shared/lib/spring-3" webAppMount="/WEB-INF/lib" />
        <PostResources className="org.apache.catalina.webresources.DirResourceSet"
                       base="/usr/shared/classes" webAppMount="/WEB-INF/classes" />
    </Resources>
</Context>

Don't forget to put the corresponding context.xml into the META-INF of your webapp.

For the jetty as well as other containers the same technique may be used. The only difference is in how to specify extra classpath elements for the webapp.


UPDATE The samples above does not share the loaded classes, but the idea is the same - use custom classloader. Here is just the pretty ugly sample that also tries to prevent classloader leaks during undeployment.


SharedWebappLoader

package com.foo.bar;

import org.apache.catalina.LifecycleException;
import org.apache.catalina.loader.WebappLoader;

public class SharedWebappLoader extends WebappLoader {

    private String pathID;
    private String pathConfig;

    static final ThreadLocal<ClassLoaderFactory> classLoaderFactory = new ThreadLocal<>();

    public SharedWebappLoader() {
        this(null);
    }

    public SharedWebappLoader(ClassLoader parent) {
        super(parent);
        setLoaderClass(SharedWebappClassLoader.class.getName());
    }

    public String getPathID() {
        return pathID;
    }

    public void setPathID(String pathID) {
        this.pathID = pathID;
    }

    public String getPathConfig() {
        return pathConfig;
    }

    public void setPathConfig(String pathConfig) {
        this.pathConfig = pathConfig;
    }

    @Override
    protected void startInternal() throws LifecycleException {
        classLoaderFactory.set(new ClassLoaderFactory(pathConfig, pathID));
        try {
            super.startInternal();
        } finally {
            classLoaderFactory.remove();
        }
    }

}

SharedWebappClassLoader

package com.foo.bar;

import org.apache.catalina.LifecycleException;
import org.apache.catalina.loader.ResourceEntry;
import org.apache.catalina.loader.WebappClassLoader;

import java.net.URL;

public class SharedWebappClassLoader extends WebappClassLoader {

    public SharedWebappClassLoader(ClassLoader parent) {
        super(SharedWebappLoader.classLoaderFactory.get().create(parent));
    }

    @Override
    protected ResourceEntry findResourceInternal(String name, String path) {
        ResourceEntry entry = super.findResourceInternal(name, path);
        if(entry == null) {
            URL url = parent.getResource(name);
            if (url == null) {
                return null;
            }

            entry = new ResourceEntry();
            entry.source = url;
            entry.codeBase = entry.source;
        }
        return entry;
    }

    @Override
    public void stop() throws LifecycleException {
        ClassLoaderFactory.removeLoader(parent);
    }
}

ClassLoaderFactory

package com.foo.bar;

import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;

public class ClassLoaderFactory {

    private static final class ConfigKey {
        private final String pathConfig;
        private final String pathID;
        private ConfigKey(String pathConfig, String pathID) {
            this.pathConfig = pathConfig;
            this.pathID = pathID;
        }
        @Override
        public boolean equals(Object o) {
            if (this == o) return true;
            if (o == null || getClass() != o.getClass()) return false;

            ConfigKey configKey = (ConfigKey) o;

            if (pathConfig != null ? !pathConfig.equals(configKey.pathConfig) : configKey.pathConfig != null)
                return false;
            if (pathID != null ? !pathID.equals(configKey.pathID) : configKey.pathID != null) return false;

            return true;
        }

        @Override
        public int hashCode() {
            int result = pathConfig != null ? pathConfig.hashCode() : 0;
            result = 31 * result + (pathID != null ? pathID.hashCode() : 0);
            return result;
        }
    }

    private static final Map<ConfigKey, ClassLoader> loaders = new HashMap<>();
    private static final Map<ClassLoader, ConfigKey> revLoaders = new HashMap<>();
    private static final Map<ClassLoader, Integer> usages = new HashMap<>();

    private final ConfigKey key;

    public ClassLoaderFactory(String pathConfig, String pathID) {
        this.key = new ConfigKey(pathConfig, pathID);
    }

    public ClassLoader create(ClassLoader parent) {
        synchronized (loaders) {
            ClassLoader loader = loaders.get(key);
            if(loader != null) {
                Integer usageCount = usages.get(loader);
                usages.put(loader, ++usageCount);
                return loader;
            }

            Properties props = new Properties();
            try (InputStream is = new BufferedInputStream(new FileInputStream(key.pathConfig))) {
                props.load(is);
            } catch (IOException e) {
                throw new RuntimeException(e);
            }

            String libsStr = props.getProperty(key.pathID);
            String[] libs = libsStr.split(File.pathSeparator);
            URL[] urls = new URL[libs.length];
            try {
                for(int i = 0, len = libs.length; i < len; i++) {
                    urls[i] = new URL(libs[i]);
                }
            } catch (MalformedURLException e) {
                throw new RuntimeException(e);
            }

            loader = new URLClassLoader(urls, parent);
            loaders.put(key, loader);
            revLoaders.put(loader, key);
            usages.put(loader, 1);

            return loader;
        }
    }

    public static void removeLoader(ClassLoader parent) {
        synchronized (loaders) {
            Integer val = usages.get(parent);
            if(val > 1) {
                usages.put(parent, --val);
            } else {
                usages.remove(parent);
                ConfigKey key = revLoaders.remove(parent);
                loaders.remove(key);
            }
        }
    }

}

context.xml of the first app

<Context>
    <Loader className="com.foo.bar.SharedWebappLoader"
            pathConfig="${catalina.base}/conf/shared.properties"
            pathID="commons_2_1"/>
</Context>

context.xml of the second app

<Context>
    <Loader className="com.foo.bar.SharedWebappLoader"
            pathConfig="${catalina.base}/conf/shared.properties"
            pathID="commons_2_6"/>
</Context>

$TOMCAT_HOME/conf/shared.properties

commons_2_1=file:/home/xxx/.m2/repository/commons-lang/commons-lang/2.1/commons-lang-2.1.jar
commons_2_6=file:/home/xxx/.m2/repository/commons-lang/commons-lang/2.6/commons-lang-2.6.jar
szhem
  • 4,672
  • 2
  • 18
  • 30
  • At first blush this sounded really good, but on second thought I question whether it will address my problem. Won't this load the library with a separate classloader for each web application? If so, then this is no better than including the libraries in the WAR file -- they still use extra memory for every deployed WAR. – mcherm Sep 29 '14 at 13:49
  • That's true - loaded classes will not be shared with VirtualWebappLocader and Pre- & Post- Resources. To share the loaded classes the idea is pretty similar - you need to implement custom classloader. The answer is updated with the sample just show that it's possible. – szhem Sep 30 '14 at 00:01
1

I was able to implement this for Tomcat (Tested on Tomcat 7.0.52). My solution involves implementing custom version of WebAppLoader which extends standard Tomcat's WebAppLoader. Thanks to this solution you can pass custom classloader to load classes for each of web application.

To utilize this new loader you need to declare it for each application (either in Context.xml file placed in each war or in Tomcat's server.xml file). This loader takes an extra custom parameter webappName which is later passed to LibrariesStorage class to determine which libraries should be used by which application.

    <Context  path="/pl-app" >
        <Loader className="web.DynamicWebappLoader" webappName="pl-app"/>
    </Context>      
    <Context  path="/my-webapp" >
        <Loader className="web.DynamicWebappLoader" webappName="myApplication2"/>
    </Context>

Once this is defined you need to install this DynamicWebappLoader to Tomcat. To do this copy all copiled classes to lib directory of Tomcat (so you should have following files [tomcat dir]/lib/web/DynamicWebappLoader.class, [tomcat dir]/lib/web/LibrariesStorage.class, [tomcat dir]/lib/web/LibraryAndVersion.class, [tomcat dir]/lib/web/WebAppAwareClassLoader.class).

You need also to download xbean-classloader-4.0.jar and place it in Tomcat's lib dir (so you should have [tomcat dir]/lib/xbean-classloader-4.0.jar. NOTE:xbean-classloader provides special implementation of classloader (org.apache.xbean.classloader.JarFileClassLoader) which allowes to load needed jars at runtime.

Main trick is made in LibraryStorgeClass (full implementation is at the end). It stores a mapping between each application (defined by webappName) and libraries which this application is allowed to load. In current implementation this is hardcoded, but this can be rewritten to dynamically generate list of libs needed by each application. Each library has its own instance of JarFileClassLoader which ensures that each library is only loaded one time (the mapping between library and its classloader is stored in static field "libraryToClassLoader", so this mapping is the same for every web application because of static nature of the field)

class LibrariesStorage {
    private static final String JARS_DIR = "D:/temp/idea_temp_proj2_/some_jars";

  private static Map<LibraryAndVersion, JarFileClassLoader> libraryToClassLoader = new HashMap<>();

  private static Map<String, List<LibraryAndVersion>> webappLibraries = new HashMap<>();

  static {
    try {
      addLibrary("commons-lang3", "3.3.2", "commons-lang3-3.3.2.jar"); // instead of this lines add some intelligent directory scanner which will detect all jars and their versions in JAR_DIR
      addLibrary("commons-lang3", "3.3.1", "commons-lang3-3.3.1.jar");
      addLibrary("commons-lang3", "3.3.0", "commons-lang3-3.3.0.jar");

      mapApplicationToLibrary("pl-app", "commons-lang3", "3.3.2"); // instead of manually mapping application to library version, some more intelligent code should be here (for example you can scann Web-Inf/lib of each application and detect needed jars

      mapApplicationToLibrary("myApplication2", "commons-lang3", "3.3.0");

     (...)    
 } 

In above example, suppose that in directory with all the jars (defined here by JARS_DIR) we have only a commons-lang3-3.3.2.jar file. This would mean that application identified by "pl-app" name (the name comes from webappName attribute in tag in Context.xml as mentioned above) will be able to load classes from commons-lang jar. Application identified by "myApplication2" will get ClassNotFoundException at this point because it has access only to commons-lang3-3.3.0.jar, but this file is not present in JARS_DIR directory.

Full implementation here:

package web;

import org.apache.catalina.loader.WebappLoader;
import org.apache.xbean.classloader.JarFileClassLoader;
import java.io.File;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;


public class DynamicWebappLoader extends WebappLoader {
  private String webappName;
  private WebAppAwareClassLoader webAppAwareClassLoader;

  public static final ThreadLocal lastCreatedClassLoader = new ThreadLocal();

  public DynamicWebappLoader() {
    super(new WebAppAwareClassLoader(Thread.currentThread().getContextClassLoader()));

    webAppAwareClassLoader = (WebAppAwareClassLoader) lastCreatedClassLoader.get(); // unfortunately I did not find better solution to access new instance of WebAppAwareClassLoader created in previous line so I passed it via thread local
    lastCreatedClassLoader.remove();

  }

  // (this method is called by Tomcat because of Loader attribute in Context.xml - <Context> <Loader className="..." webappName="myApplication2"/> )
  public void setWebappName(String name) {
    System.out.println("Setting webapp name: " + name);
    this.webappName = name;
    webAppAwareClassLoader.setWebAppName(name); // pass web app name to ClassLoader 
  }


}


class WebAppAwareClassLoader extends ClassLoader {
  private String webAppName;

  public WebAppAwareClassLoader(ClassLoader parent) {
    super(parent);
    DynamicWebappLoader.lastCreatedClassLoader.set(this); // store newly created instance in ThreadLocal .. did not find better way to access the reference later in code
  }

  @Override
  public Class<?> loadClass(String className) throws ClassNotFoundException {
    System.out.println("Load class: " + className + " for webapp: " + webAppName);
    try {
      return LibrariesStorage.loadClassForWebapp(webAppName, className);
    } catch (ClassNotFoundException e) {
      System.out.println("JarFileClassLoader did not find class: " + className + " " + e.getMessage());
      return super.loadClass(className);
    }

  }

  public void setWebAppName(String webAppName) {
    this.webAppName = webAppName;
  }
}

class LibrariesStorage {
  private static final String JARS_DIR = "D:/temp/idea_temp_proj2_/some_jars";

  private static Map<LibraryAndVersion, JarFileClassLoader> libraryToClassLoader = new HashMap<>();

  private static Map<String, List<LibraryAndVersion>> webappLibraries = new HashMap<>();

  static {
    try {
      addLibrary("commons-lang3", "3.3.2", "commons-lang3-3.3.2.jar"); // instead of this lines add some intelligent directory scanner which will detect all jars and their versions in JAR_DIR
      addLibrary("commons-lang3", "3.3.1", "commons-lang3-3.3.1.jar");
      addLibrary("commons-lang3", "3.3.0", "commons-lang3-3.3.0.jar");

      mapApplicationToLibrary("pl-app", "commons-lang3", "3.3.2"); // instead of manually mapping application to library version, some more intelligent code should be here (for example you can scann Web-Inf/lib of each application and detect needed jars
      mapApplicationToLibrary("myApplication2", "commons-lang3", "3.3.0");

    } catch (MalformedURLException e) {
      throw new RuntimeException(e.getMessage(), e);
    }

  }

  private static void mapApplicationToLibrary(String applicationName, String libraryName, String libraryVersion) {
    LibraryAndVersion libraryAndVersion = new LibraryAndVersion(libraryName, libraryVersion);
    if (!webappLibraries.containsKey(applicationName)) {
      webappLibraries.put(applicationName, new ArrayList<LibraryAndVersion>());
    }
    webappLibraries.get(applicationName).add(libraryAndVersion);
  }

  private static void addLibrary(String libraryName, String libraryVersion, String filename)
                          throws MalformedURLException {
    LibraryAndVersion libraryAndVersion = new LibraryAndVersion(libraryName, libraryVersion);
    URL libraryLocation = new File(JARS_DIR + File.separator + filename).toURI().toURL();

    libraryToClassLoader.put(libraryAndVersion,
      new JarFileClassLoader("JarFileClassLoader for lib: " + libraryAndVersion,
        new URL[] { libraryLocation }));
  }

  private LibrariesStorage() {
  }


  public static Class<?> loadClassForWebapp(String webappName, String className) throws ClassNotFoundException {
    System.out.println("Loading class: " + className + " for web application: " + webappName);

    List<LibraryAndVersion> webappLibraries = LibrariesStorage.webappLibraries.get(webappName);
    for (LibraryAndVersion libraryAndVersion : webappLibraries) {
      JarFileClassLoader libraryClassLoader = libraryToClassLoader.get(libraryAndVersion);

      try {
        return libraryClassLoader.loadClass(className); // ok current lib contained class to load
      } catch (ClassNotFoundException e) {
        // ok.. continue in loop... try to load the class from classloader connected to next library
      }
    }

    throw new ClassNotFoundException("Class " + className + " was not found in any jar connected to webapp: " +
      webappLibraries);

  }

}


class LibraryAndVersion {
  private final String name;
  private final String version;

  LibraryAndVersion(String name, String version) {
    this.name = name;
    this.version = version;
  }

  @Override
  public boolean equals(Object o) {
    if (this == o) {
      return true;
    }
    if ((o == null) || (getClass() != o.getClass())) {
      return false;
    }

    LibraryAndVersion that = (LibraryAndVersion) o;

    if ((name != null) ? (!name.equals(that.name)) : (that.name != null)) {
      return false;
    }
    if ((version != null) ? (!version.equals(that.version)) : (that.version != null)) {
      return false;
    }

    return true;
  }

  @Override
  public int hashCode() {
    int result = (name != null) ? name.hashCode() : 0;
    result = (31 * result) + ((version != null) ? version.hashCode() : 0);
    return result;
  }

  @Override
  public String toString() {
    return "LibraryAndVersion{" +
      "name='" + name + '\'' +
      ", version='" + version + '\'' +
      '}';
  }
}
walkeros
  • 4,736
  • 4
  • 35
  • 47
  • That sounds really good. I need to think about this carefully, and probably go try it out, but it sounds like this is a functioning solution to my problem. Did you write this up to answer my question, or is this something you have actually used previously? – mcherm Sep 30 '14 at 00:55
  • 1
    I wrote this up especially for your question, but I come up with this basing on my recent use case: I wanted to deploy app on EmbedTomcat in my unit tests. I pointed docroot to source folder in my IDE but obviously there were no jars in WEB-INF/lib (this was my maven src dir: src/main/webapp in IDE). I needed to somehow modify the webapp classloader to load jars from another directory so I used JarFileClassLoader).It works for some time already on our CI without problems. Tomcat loads all the classes and even the jsp TLDs from jars from external dir.Threfore I'm almost sure that it will work:) – walkeros Sep 30 '14 at 05:20
1

JBoss has a framework called Modules that solves this problem. You can save the shared library with its version and reference it from your war-file.

I have no idea if it works on Tomcat, but it works as a charm on Wildfly.

Kurt Du Bois
  • 7,550
  • 4
  • 25
  • 33