How to get ZK webfragment working with embedded Jetty 9?

Question

This minimal embedded Jetty project starts up correctly, scans for annotations and finds and maps the annotated TestServlet.

Project structure:

|-src/main/java/test
|  |-Test.java
|-webapp/
|  |-test.zul
|-pom.xml

Test.java:

package test;

import java.io.File;
import java.io.IOException;
import java.net.URI;

import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.eclipse.jetty.annotations.AnnotationConfiguration;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.util.resource.Resource;
import org.eclipse.jetty.webapp.Configuration;
import org.eclipse.jetty.webapp.FragmentConfiguration;
import org.eclipse.jetty.webapp.MetaInfConfiguration;
import org.eclipse.jetty.webapp.WebAppContext;
import org.eclipse.jetty.webapp.WebInfConfiguration;
import org.eclipse.jetty.webapp.WebXmlConfiguration;

public class Test {
    public static void main(String[] args) throws Exception {
        Server server = new Server(8080);
        WebAppContext webapp = new WebAppContext();
        webapp.setContextPath("/test");
        webapp.setBaseResource(Resource.newResource(new File("webapp").getCanonicalFile()));
        // https://www.eclipse.org/jetty/documentation/jetty-9/index.html#configuring-webapps
        // the order is important
        webapp.setConfigurations(new Configuration[] { //
            new WebInfConfiguration(), //
            new WebXmlConfiguration(), //
            new MetaInfConfiguration(), //
            new FragmentConfiguration(), //
            // new EnvConfiguration(), // not needed
            // new PlusConfiguration(), // not needed
            new AnnotationConfiguration(), //
            // new JettyWebXmlConfiguration(), // no jetty-web.xml
        });
        webapp.setAttribute("org.eclipse.jetty.server.webapp.ContainerIncludeJarPattern", ".*");
        server.setHandler(webapp);
        server.setDumpAfterStart(true);
        server.start();
        java.awt.Desktop.getDesktop().browse(new URI("http://localhost:8080/test/TestServlet")) /* working */;
        java.awt.Desktop.getDesktop().browse(new URI("http://localhost:8080/test/test.zul")) /* not working */;
    }
    
    @WebServlet(urlPatterns = {"/TestServlet"})
    public static final class TestServlet extends HttpServlet {
        @Override
        protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
            resp.getWriter().write("Test 1");
        }
    }
}

pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>test</groupId>
    <artifactId>test</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <dependencies>
        <dependency>
            <groupId>org.eclipse.jetty</groupId>
            <artifactId>jetty-annotations</artifactId>
            <version>9.4.30.v20200611</version>
        </dependency>
        <dependency>
            <groupId>org.zkoss.zk</groupId>
            <artifactId>zkbind</artifactId>
            <version>9.6.0.1</version>
        </dependency>
    </dependencies>
</project>

test.zul:

<zk><label value="hello"/></zk>

The zk web fragment seems to be 'noted' somehow:

|  +@ org.eclipse.jetty.webFragments.cache = java.util.concurrent.ConcurrentHashMap@bb35baa5{size=30}
|  |  +@ file:///C:/Users/r.hoehener/.m2/repository/org/zkoss/common/zcommon/9.6.0.1/zcommon-9.6.0.1.jar = org.eclipse.jetty.util.resource.EmptyResource@3fc39309
|  |  +@ file:///C:/Users/r.hoehener/.m2/repository/org/eclipse/jetty/jetty-annotations/9.4.30.v20200611/jetty-annotations-9.4.30.v20200611.jar = org.eclipse.jetty.util.resource.EmptyResource@3fc39309
|  |  +@ file:///C:/Users/r.hoehener/.m2/repository/org/eclipse/jetty/jetty-io/9.4.30.v20200611/jetty-io-9.4.30.v20200611.jar = org.eclipse.jetty.util.resource.EmptyResource@3fc39309
|  |  +@ file:///C:/Users/r.hoehener/.m2/repository/org/apache-extras/beanshell/bsh/2.0b6/bsh-2.0b6.jar = org.eclipse.jetty.util.resource.EmptyResource@3fc39309
|  |  +@ file:///C:/Users/r.hoehener/.m2/repository/org/zkoss/zk/zkwebfragment/9.6.0.1/zkwebfragment-9.6.0.1.jar = jar:file:///C:/Users/r.hoehener/.m2/repository/org/zkoss/zk/zkwebfragment/9.6.0.1/zkwebfragment-9.6.0.1.jar!/META-INF/web-fragment.xml
...

But the test.zul is displayed as plain text. The ZK engine is not getting initialized.

Any ideas why?

Edit: In defense of the way I do the configuration: This is straight from the 9.x docs, which say 'You have a number of options for how to make Jetty use a different list of Configurations.', including 'Setting the list directly on the WebAppContext':

<Configure class="org.eclipse.jetty.webapp.WebAppContext">
  <Set name="war"><SystemProperty name="jetty.base" default="."/>/webapps/my-cool-webapp</Set>
  <Set name="configurationClasses">
    <Array type="java.lang.String">
      <Item>org.eclipse.jetty.webapp.WebInfConfiguration</Item>
      <Item>org.eclipse.jetty.webapp.WebXmlConfiguration</Item>
      <Item>org.eclipse.jetty.webapp.MetaInfConfiguration</Item>
      <Item>org.eclipse.jetty.webapp.FragmentConfiguration</Item>
      <Item>org.eclipse.jetty.plus.webapp.EnvConfiguration</Item>
      <Item>org.eclipse.jetty.plus.webapp.PlusConfiguration</Item>
      <Item>org.eclipse.jetty.annotations.AnnotationConfiguration</Item>
      <Item>org.eclipse.jetty.webapp.JettyWebXmlConfiguration</Item>
    </Array>
  </Set>
</Configure>

Answer 1

the way to add a configuration is:

Configuration.ClassList classList = Configuration.ClassList.setServerDefault(server);
classList.addAfter(
                "org.eclipse.jetty.webapp.FragmentConfiguration",
                "org.eclipse.jetty.annotations.AnnotationConfiguration");

Update:

I have tried

    <dependency>
        <groupId>org.eclipse.jetty</groupId>
        <artifactId>jetty-annotations</artifactId>
        <version>10.0.7</version>
    </dependency>

The zul works without adding any configuration.

Answer 2

First, don't use Jetty 9.4.30, it's subject to a few security advisories now.

See: https://www.eclipse.org/jetty/security_reports.php

Use at least use Jetty 9.4.44.v20210927.

Next, check your Jetty Server dump for the zk servlets ...

• org.zkoss.zk.au.http.DHtmlUpdateServlet
• org.zkoss.zk.ui.http.DHtmlLayoutServlet

If those are present in your WebAppContext dump then your zkwebfragment-<ver>.jar was discovered and loaded by Jetty properly. At this point, what you have left to do is how to properly configure for your zk lib using the zk techniques (you can ignore Jetty specific details from here on).

If they are not present, then first make sure your own webapp is using Servlet 3.0 (declared in your WEB-INF/web.xml) or newer for proper Web Fragment support (older Servlet specs don't support Web Fragment).

Next, make sure the zkwebfragment-<ver>.jar is present on the WebApp classloader, as no web-fragments will load from any other classloader per spec, not even the application / server / container classloaders.

If you still don't see them, then go back to adjusting the default Configuration list, not the hardcoded list you have in your code snippet. (your list is missing required Configurations, and is in the wrong order for success, don't alter the default list, don't set the list on the webapp, only alter the server level defaults).

Ask yourself, what does zk require? (eg: if it needs jndi, then you need the jndi specific configuration piece too).

If you are not stuck on Java 8, please use Jetty 10, as the entire Configuration layer was reworked to no longer allow bad configurations (in fact the old setConfiguration() methods are not even there, just the existence of support jars is enough to flag that you want that support and enable it, in the right place, with the right parent dependencies).

Answer 3

So, as Joakim pointed out, the Servlet Specification 3.1 says:

If a framework wants its META-INF/web-fragment.xml honored in such a way that it augments a web application's web.xml, the framework must be bundled within the web application's WEB-INF/lib directory. [...] In other words, only JAR files bundled in a web application's WEB-INF/lib directory, but not those higher up in the class loading delegation chain, need to be scanned for web-fragment.xml

Therefore, I must ensure that zkwebfragments.jar is present on the WebAppContext's classpath.

I changed my deployment strategy to look like this:

/myapp/
  |-bin/
  |  |- project jar (including annotated servlets and listeners)
  |  |- all maven dependency jars (including jetty and zk libs)
  |-webapp/
  |  |-WEB-INF/
  |  |  |-lib/ (EMTPY!)
  |  |  |-web.xml
  |  |  |-zk.xml
  |  |-index.zul
  |  |-other static resources
  |-data/
     |-application data (docker mounted host volume)

In the main class, adding the zkwebfragment.jar to the WebAppContext classpath:

Server server = new Server(port);
WebAppContext webapp = new WebAppContext();
webapp.setContextPath("/myapp");
webapp.setBaseResource(Resource.newResource(new File(isDocker ? "/myapp/webapp" : "./webapp").getCanonicalFile()));
// order important: https://www.eclipse.org/jetty/documentation/jetty-9/index.html#configuring-webapps
webapp.setConfigurations(new Configuration[] { //
    new WebInfConfiguration(), //
    new WebXmlConfiguration(), //
    new MetaInfConfiguration(), //
    new FragmentConfiguration(), //
    new AnnotationConfiguration(), //
});
// servlet spec: fragments are only loaded if they are bundled in WEB-INF/lib
webapp.setExtraClasspath(Arrays.asList(((URLClassLoader) MyAppMain.class.getClassLoader()).getURLs()).stream().filter(
    u -> u.toString().contains("zkwebfragment")).findAny().get().toString());
// scan for annotations in the container classpath (AppClassLoader)
webapp.setAttribute("org.eclipse.jetty.server.webapp.ContainerIncludeJarPattern", ".*");
server.setHandler(webapp);
server.setDumpAfterStart(true);
server.start();
   

When running the main class from an IDE, all project classes and all maven dependencies are automatically added to the AppClassLoader.

When running as docker container, the application is started with:

ENTRYPOINT ["java", "-cp", "/myapp/bin/*", "myapp.MyAppMain"]

which also adds the project jar and all maven dependency jars to the AppClassLoader.

By moving all jars out of /myapp/webapp/META-INF/lib/* to /myapp/bin/*, all jars exist only once in the classloader hierarchy, which also solves the ZK replicate resources warnings. Except for the zkwebfragment.jar, which exists twice. Jetty dump excerpts:

| |  +> WebAppClassLoader{1509563803}@59fa1d9b
| |  |  +> URLs size=1
| |  |  |  +> file:/myapp/bin/zkwebfragment-9.6.0.1.jar
| |  |  +> sun.misc.Launcher$AppClassLoader@2ff4acd0

and further down:

| +> sun.misc.Launcher$AppClassLoader@2ff4acd0
|    +> URLs size=52
|    |  +> file:/myapp/bin/myapp.jar
|    |  +> file:/myapp/bin/jetty-annotations-9.4.44.v20210927.jar
|    |  +> file:/myapp/bin/zkwebfragment-9.6.0.1.jar
|    |  +> ...
|    +> sun.misc.Launcher$ExtClassLoader@4ee285c6
|       +> URLs size=9
|          +> file:/opt/java/openjdk/lib/ext/sunpkcs11.jar
|          +> ...

None of this would be necessary if I were using a Jetty installation and dropping my application bundled as a standard WAR. But I really wanted to try out this plain Java approach (from the Jetty docs):

Jetty has a slogan, "Don’t deploy your application in Jetty, deploy Jetty in your application!" What this means is that as an alternative to bundling your application as a standard WAR to be deployed in Jetty, Jetty is designed to be a software component that can be instantiated and used in a Java program just like any POJO.