How can I get Jetty to scan the Maven dependency jar files in the class path?

Question

Eclipse automatically adds all Maven dependencies to the class path.

How can I get Jetty to scan the jar files on the class path for web fragments, when started from Eclipse?

When packaged and placed in webapp/WEB-INF/lib/*.jar, the scanning works.

Server server = new Server(8080);

WebAppContext webapp = new WebAppContext();
webapp.setContextPath("/myapp");
webapp.setBaseResource(Resource.newResource(new File("webapp")));

// https://stackoverflow.com/questions/11768113/cant-get-jetty-to-scan-for-annotated-classes
webapp.setConfigurations(new Configuration[] { //
    new AnnotationConfiguration(), // @WebServlet, @WebListener...
    new WebXmlConfiguration(), // webapp/WEB-INF/web.xml
    new WebInfConfiguration(), // ?
    new MetaInfConfiguration(), // ?
    new FragmentConfiguration(), // e.g. zkwebfragment-9.6.0.1.jar!/META-INF/web-fragment.xml
});

if(RUNNING_FROM_IDE)
{
  // add project classes to classpath (e.g. target/classes)
  webapp.getMetaData().setWebInfClassesDirs(Arrays.asList(Resource.newResource(MyAppMainClass.class.getProtectionDomain().getCodeSource().getLocation())));

  // how to add maven dependencies?
  // webapp.getMetaData().addWebInfJar(???);
}

server.setHandler(webapp);
server.start();

score 1 · Accepted Answer · answered Dec 20 '21 at 14:03

Don't manually set the configurations, that's the first mistake.

Your effort basically broke the configurations, as you didn't respect the original default configurations, or the delicate order requirements.

This is wrong way, by setting the webapp configurations directly.

webapp.setConfigurations(new Configuration[] { //
    new AnnotationConfiguration(), // @WebServlet, @WebListener...
    new WebXmlConfiguration(), // webapp/WEB-INF/web.xml
    new WebInfConfiguration(), // ?
    new MetaInfConfiguration(), // ?
    new FragmentConfiguration(), // e.g. zkwebfragment-9.6.0.1.jar!/META-INF/web-fragment.xml
});

This is the correct way, by adjusting the default Configuration list at the Server level.

Configuration.ClassList classlist = Configuration.ClassList
  .setServerDefault(server);
classlist.addBefore(
  "org.eclipse.jetty.webapp.JettyWebXmlConfiguration",
  "org.eclipse.jetty.annotations.AnnotationConfiguration");

This should be enough.

You can see the default configuration order on the server dump.

server.setDumpAfterStart(true);
server.start();

// and then on the output (see your logger) ...
|  |  |  +> Configurations Servlet Dump WebApp@4c163e3 size=5
|  |  |  |  +> org.eclipse.jetty.webapp.WebInfConfiguration@5e403b4a
|  |  |  |  +> org.eclipse.jetty.webapp.WebXmlConfiguration@5117dd67
|  |  |  |  +> org.eclipse.jetty.webapp.MetaInfConfiguration@5be49b60
|  |  |  |  +> org.eclipse.jetty.webapp.FragmentConfiguration@2931522b
|  |  |  |  +> org.eclipse.jetty.webapp.JettyWebXmlConfiguration@7674b62c
|  |  |  +> Handler attributes Servlet Dump WebApp@4c163e3 size=3

This dump feature is useful, as it will likely tell you that the the $HOME/.m2/repository jar files are not present in the webapp's own classpath.

|  |  |  +> WebAppClassLoader{Servlet Dump WebApp}@5fb759d6
|  |  |  |  +> URLs size=1
|  |  |  |  |  +> file:/tmp/jetty-0_0_0_0-8080-ROOT_war-_-any-15598896298108484560/webapp/WEB-INF/classes/

If you don't see your maven repository files here, then it's time for you to configure your WebAppContext.setExtraClasspath(String) to include them before you start your webapp.

Another feature to be aware of are the 2 scan regexes used to identify which content (class files) are scanned. One for the webapp classes, and one for the server classes. Both of these are relevant for the discovery of anything in the servlet spec, be it annotated listeners, annotated servlets, web fragments, servlet container initializers, jsp level details, websocket level details, etc)

Both are set as attributes on the WebAppContext.setAttribute(String key, String value) if you want to configure them.

By default, these values are empty, meaning all jars are scanned.

// This one controls server/container level discovery.

// An example of limited scanning would be
webappContext.setAttribute(
  "org.eclipse.jetty.server.webapp.ContainerIncludeJarPattern"
  ".*/[^/]*servlet-api-[^/]*\\.jar$|.*[^/]*jstl-.*\\.jar$|.*/[^/]*taglibs.*\\.jar$");


// This one controls webapp specific discovery.
webappContext.setAttribute(
  "org.eclipse.jetty.server.webapp.WebInfIncludeJarPattern"
  ".*/[^/]*foo-[^/]*\\.jar$|.*/[^/]*bar.*\\.jar$");

An example of this would be ...

You have a webapp with ...

WEB-INF/lib/
           foo-api-1.2.3.jar
           foo-1.2.3.jar
           bar-0.9.jar
           boo-1.0.jar

And defined the webapp scanning like ...

webappContext.setAttribute(
    "org.eclipse.jetty.server.webapp.WebInfIncludeJarPattern",
    ".*/.*foo-api-[^/]\.jar$|./.*bar-[^/]\.jar$|./.*wibble[^/]*\.jar$");

Then the following files would match and be scanned:

WEB-INF/lib/
           foo-api-1.2.3.jar
           bar-0.9.jar

I tried to follow your instructions (see my other answer), but now I see that basically all my dependencies are duplicated: In the WebAppClassLoader and in the AppClassLoader. — Reto Höhener, Dec 20 '21 at 15:23
The docs (https://www.eclipse.org/jetty/documentation/jetty-9/index.html#configuring-webapps) give an explicit example of 'Setting the list directly on the WebAppContext', so I don't think that approach was completely wrong. — Reto Höhener, Dec 20 '21 at 16:33

Reto Höhener · Answer 2 · 2021-12-20T13:49:30.740

I got it to work somehow. Still wondering if there is a more standard way to achieve this:

// launching from IDE: manually add WEB-INF/classes and WEB-INF/lib/*.jar entries
webapp.getMetaData().setWebInfClassesDirs(Arrays.asList(Resource.newResource(MyAppMainClass.class.getProtectionDomain().getCodeSource().getLocation())));
for(URL url : ((URLClassLoader) MyAppMainClass.class.getClassLoader()).getURLs()) {
  if(url.getPath().endsWith(".jar")) {
    webapp.getMetaData().addWebInfJar(Resource.newResource(url));
  }
}

I also found MavenWebInfConfiguration in the jetty-maven-plugin. I uses:

((WebAppClassLoader)  webapp.getClassLoader()).addClassPath(...)

Don't mess with the MetaData like that, that's bad juju and can easily be undone by processes during webapp initialization. See above correct techniques. — Joakim Erdfelt, Dec 20 '21 at 14:03

Reto Höhener · Answer 3 · 2021-12-20T15:23:37.560

Based on Joakim's answer, I now have:

Server server = new Server(port);
Configuration.ClassList classlist = Configuration.ClassList.setServerDefault(server);
classlist.addAfter(JettyWebXmlConfiguration.class.getName(), AnnotationConfiguration.class.getName());

WebAppContext webapp = new WebAppContext();
webapp.setContextPath("/myapp");
webapp.setBaseResource(Resource.newResource(new File(isRunningFromIde ? "./webapp" : "/path/to/webapp").getCanonicalFile()));    
webapp.setWelcomeFiles(new String[] { "index.html" });
//    webapp.setInitParameter(..., ...));
//    webapp.setAttribute("org.eclipse.jetty.server.webapp.ContainerIncludeJarPattern", "");
//    webapp.setAttribute("org.eclipse.jetty.server.webapp.WebInfIncludeJarPattern", "");

if(isRunningFromIde)
{
  List<String> extraClassPath = new ArrayList<>();
  // target/classes
  extraClassPath.add(Paths.get(MyAppMain.class.getProtectionDomain().getCodeSource().getLocation().toURI()).toString());
  // maven dependencies
  for(URL url : ((URLClassLoader) MyAppMain.class.getClassLoader()).getURLs()) {
    if(url.getPath().endsWith(".jar")) {
      extraClassPath.add(Paths.get(url.toURI()).toString());
    }
  }
  webapp.setExtraClasspath(extraClassPath.stream().collect(Collectors.joining(";")));
}

server.setHandler(webapp);
server.setDumpAfterStart(true);
server.start();

Thanks to the server dump tip, I think I now also understand why I am getting replicate resource warnings from ZK. The WebAppClassLoader and the AppClassLoader both contain the same entries. The IDE launch because of the above extra classpath. The docker container probably because I start it with java -cp <webapp>/WEB-INF/lib/* MyAppMain.class.

|  +> WebAppClassLoader{403716510}@1810399e
|  |  +> URLs size=41
|  |  |  +> file:<maven-repo-or-webapp-dir>/WEB-INF/lib/zk-9.6.0.1.jar
|  |  |  ...
|  |  +> sun.misc.Launcher$AppClassLoader@4e0e2f2a
|  |  |  +> URLs size=41
|  |  |  |  +> file:<maven-repo-or-webapp-dir>/WEB-INF/lib/zk-9.6.0.1.jar
|  |  |  |  ...

You probably don't want to add ALL of the jars from the URLClassLoader, but specific ones. Replace the entire block inside `if(isRunningFromIde)` with just `webapp.setParentLoaderPriority(true)` and you have the same end result without the extraclasspath step — Joakim Erdfelt, Dec 20 '21 at 16:52
Indeed, I just discovered that parent class loader property. I was fiddling with it just now but cannot get jetty to scan it for annotations. I was hoping that this parent class loader would be the 'container class path', and also set the ContainerIncludeJarPattern to `.*`, but still no luck :( — Reto Höhener, Dec 20 '21 at 16:55
Hm, I was wrong. It *does* scan for annotations, just the zk web fragments don't seem to load correctly. — Reto Höhener, Dec 20 '21 at 16:59

score 0 · Answer 4 · answered Dec 20 '21 at 18:33

Attempt 3 (also based on Joakim's answer): Setting the ContainerIncludeJarPattern to scan everything that is already on the classpath. No need for any additional class path modifications.

    Server server = new Server(8080);
    WebAppContext webapp = new WebAppContext();
    webapp.setContextPath("/test");
    webapp.setBaseResource(Resource.newResource(new File("webapp").getCanonicalFile()));
    // https://www.eclipse.org/jetty/documentation/jetty-9/index.html#configuring-webapps
    // order apparently important
    webapp.setConfigurations(new Configuration[] { //
        new WebInfConfiguration(), //
        new WebXmlConfiguration(), //
        new MetaInfConfiguration(), //
        new FragmentConfiguration(), //
        new AnnotationConfiguration(), //
    });
    webapp.setAttribute("org.eclipse.jetty.server.webapp.ContainerIncludeJarPattern", ".*");
    server.setHandler(webapp);
    server.setDumpAfterStart(true);
    server.start();

My other problems seem to be ZK-related, so I started a new question for that.

How can I get Jetty to scan the Maven dependency jar files in the class path?

4 Answers4