Embedded Jetty upgrade issues from 6.1.7 to 9.4.43

Question

I have used jetty.xml for configuration in older jetty version which looks something like this:

<Configure id="Server" class="org.mortbay.jetty.Server">
    <Set name="ThreadPool">
        <New class="org.mortbay.thread.BoundedThreadPool">
            <Set name="minThreads">10</Set>
            <Set name="maxThreads">250</Set>
            <Set name="lowThreads">25</Set>
        </New>
    </Set>

    <Call name="addConnector">
        <Arg>
            <New class="org.mortbay.jetty.nio.SelectChannelConnector">
                <Set name="host">
                    <SystemProperty name="jetty.host"/>
                </Set>
                <Set name="port">
                    <SystemProperty name="jetty.port" default="9096"/>
                </Set>
                <Set name="maxIdleTime">30000</Set>
                <Set name="Acceptors">2</Set>
                <Set name="statsOn">false</Set>
                <Set name="confidentialPort">8443</Set>
                <Set name="lowResourcesConnections">5000</Set>
                <Set name="lowResourcesMaxIdleTime">5000</Set>
            </New>
        </Arg>
    </Call>

    <Set name="handler">
        <New id="Handlers" class="org.mortbay.jetty.handler.HandlerCollection">
            <Set name="handlers">
                <Array type="org.mortbay.jetty.Handler">
                    <Item>
                        <New id="Contexts" class="org.mortbay.jetty.handler.ContextHandlerCollection"/>
                    </Item>
                    <!-- Disabling this as it is a security risk (by exposing all contexts) in production-->
                    <!--Item>
                        <New id="DefaultHandler" class="org.mortbay.jetty.handler.DefaultHandler"/>
                    </Item-->
                    <Item>
                        <New id="RequestLog" class="org.mortbay.jetty.handler.RequestLogHandler"/>
                    </Item>
                </Array>
            </Set>
        </New>
    </Set>

    <Call name="addLifeCycle">
        <Arg>
            <New class="org.mortbay.jetty.deployer.ContextDeployer">
                <Set name="contexts">
                    <Ref id="Contexts"/>
                </Set>
                <Set name="configurationDir">
                    <SystemProperty name="jetty.home" default="."/>/contexts</Set>
                <Set name="scanInterval">1</Set>
            </New>
        </Arg>
    </Call>

    <New class="org.mortbay.jetty.servlet.Context">
        <Arg>
            <Ref id="Contexts"/>
        </Arg>
        <Arg>/cts/output</Arg>
        <Set name="resourceBase">
            <SystemProperty name="jetty.home" default="."/>/../cache/</Set>
        <Call name="addServlet">
            <Arg>com.myservlet.webservices.remote.FileProxyServlet</Arg>
            <Arg>/</Arg>
        </Call>
    </New>
    <Ref id="RequestLog">
        <Set name="requestLog">
            <New id="RequestLogImpl" class="org.mortbay.jetty.NCSARequestLog">
                <Set name="filename">
                    <SystemProperty name="jetty.logs" default="../logs"/>/ws.request_yyyy_mm_dd.log</Set>
                <Set name="filenameDateFormat">yyyy_MM_dd</Set>
                <Set name="retainDays">90</Set>
                <Set name="append">true</Set>
                <Set name="extended">true</Set>
                <Set name="logCookies">false</Set>
                <Set name="LogTimeZone">GMT</Set>
            </New>
        </Set>
    </Ref>
    <!-- =========================================================== -->
    <!-- extra options                                               -->
    <!-- =========================================================== -->
    <Set name="stopAtShutdown">true</Set>
    <Set name="sendServerVersion">true</Set>
    <Set name="sendDateHeader">true</Set>
    <Set name="gracefulShutdown">1000</Set>
</Configure>```

And to call this configuration following code has been written:
    <Ref id="RequestLog">
        <Set name="requestLog">
            <New id="RequestLogImpl" class="org.mortbay.jetty.NCSARequestLog">
                <Set name="filename">
                    <SystemProperty name="jetty.logs" default="../logs"/>/ws.request_yyyy_mm_dd.log</Set>
                <Set name="filenameDateFormat">yyyy_MM_dd</Set>
                <Set name="retainDays">90</Set>
                <Set name="append">true</Set>
                <Set name="extended">true</Set>
                <Set name="logCookies">false</Set>
                <Set name="LogTimeZone">GMT</Set>
            </New>
        </Set>
    </Ref>
    <!-- =========================================================== -->
    <!-- extra options                                               -->
    <!-- =========================================================== -->
    <Set name="stopAtShutdown">true</Set>
    <Set name="sendServerVersion">true</Set>
    <Set name="sendDateHeader">true</Set>
    <Set name="gracefulShutdown">1000</Set>
</Configure>

Following code has been written to call this configuration:

Server myServer;
File theConfigFile = new File( "C://jetty//etc//jetty.xml" );
XmlConfiguration theXmlConfiguration = new XmlConfiguration( theConfigFile.toURL() );
theXmlConfiguration.configure( myServer );
myServer.start();

I was trying to create similar configuration using jetty 9.4.43:

<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure.dtd">
 
<Configure id="jetty" class="org.eclipse.jetty.server.Server">
    <New id="threadPool" class="org.eclipse.jetty.util.thread.QueuedThreadPool">
        <Set name="minThreads" type="int"><Property name="jetty.threadPool.minThreads" deprecated="threads.min" default="10"/></Set>
        <Set name="maxThreads" type="int"><Property name="jetty.threadPool.maxThreads" deprecated="threads.max" default="200"/></Set>
        <Set name="reservedThreads" type="int"><Property name="jetty.threadPool.reservedThreads" default="25"/></Set>
        <Set name="idleTimeout" type="int"><Property name="jetty.threadPool.idleTimeout" deprecated="threads.timeout" default="60000"/></Set>
        <Set name="detailedDump" type="boolean"><Property name="jetty.threadPool.detailedDump" default="false"/></Set>
    </New>
    
    <Call name="addBean">
      <Arg>
        <New class="org.eclipse.jetty.util.thread.ScheduledExecutorScheduler"/>
      </Arg>
    </Call>
 
   <Set name="connectors">
    <Array type="org.eclipse.jetty.server.Connector">
      <Item>
        <New class="org.eclipse.jetty.server.ServerConnector">
          <Arg><Ref refid="jetty"/></Arg>
          <Set name="port">
            <Property name="jetty.http.port" default="9096" />
          </Set>
        </New>
      </Item>
    </Array>
  </Set>

    <Set name="handler">
      <New class="org.eclipse.jetty.server.handler.HandlerList">
        <Set name="handlers">
      <Array type="org.eclipse.jetty.server.Handler">
        <Item>
          <New class="org.eclipse.jetty.server.handler.ResourceHandler">
            <Set name="directoriesListed">true</Set>
            <Set name="resourceBase">.</Set>
          </New>
        </Item>
      </Array>
        </Set>
      </New>
    </Set>
    
    <New class="org.eclipse.jetty.servlet.ServletContextHandler">
        <Arg>
            <Ref id="Contexts"/>
        </Arg>
        <Arg>/cts/output</Arg>
        <Set name="resourceBase">
            <SystemProperty name="jetty.home" default="."/>/../cache/</Set>
        <Call name="addServlet">
            <Arg>com.myservlet.webservices.remote.FileProxyServlet</Arg>
            <Arg>/</Arg>
        </Call>
    </New>

    <Set name="stopAtShutdown"><Property name="jetty.server.stopAtShutdown" default="true"/></Set>
    <Set name="stopTimeout"><Property name="jetty.server.stopTimeout" default="5000"/></Set>
    <Set name="dumpAfterStart"><Property name="jetty.server.dumpAfterStart" deprecated="jetty.dump.start" default="true"/></Set>
    <Set name="dumpBeforeStop"><Property name="jetty.server.dumpBeforeStop" deprecated="jetty.dump.stop" default="true"/></Set>
</Configure> 

With new jetty.xml, port 9096 is exposed and all files available under that directory from where intelliJ is running is accessible but the FileProxyServlet url i.e localhost:9096/cts/output is not accessible(shows HTTP ERROR 404 Not Found). This was not the case earlier instead only servlet was working fine.

Below is the console log on running with new jetty:

System Property [DEBUG] has been deprecated! (Use org.eclipse.jetty.LEVEL=DEBUG instead) 2021-10-16 13:56:48.162:INFO::main: Logging initialized @80050ms to org.eclipse.jetty.util.log.StdErrLog [ShutdownMonitor] Not enabled (port < 0): -1 2021-10-16 13:56:48.547:INFO:oejs.Server:Jetty Web Server Start Thread: jetty-9.4.43.v20210629; built: 2021-06-30T11:07:22.254Z; git: 526006ecfa3af7f1a27ef3a288e2bef7ea9dd7e8; jvm 11.0.7+10-LTS 2021-10-16 13:56:48.650:INFO:oejs.AbstractConnector:Jetty Web Server Start Thread: Started ServerConnector@70239ac0{HTTP/1.1, (http/1.1)}{0.0.0.0:9096} Server@4db85c57{STARTING}[9.4.43.v20210629]

• STARTING += QueuedThreadPool[qtp1589931229]@5ec46cdd{STARTED,8<=8<=200,i=6,r=-1,q=0}[ReservedThreadExecutor@16627f86{s=0/2,p=0}]
• STARTED | +- org.eclipse.jetty.util.thread.ThreadPoolBudget@2d7c53b6 | += ReservedThreadExecutor@16627f86{s=0/2,p=0} - STARTED | +> threads size=8 | +> qtp1589931229-31 RUNNABLE tid=31 prio=5 @ java.base@11.0.7/java.lang.invoke.InnerClassLambdaMetafactory.getParameterSize(InnerClassLambdaMetafactory.java:522) | +> qtp1589931229-34 TIMED_WAITING tid=34 prio=5 IDLE | +> qtp1589931229-29 TIMED_WAITING tid=29 prio=5 IDLE | +> qtp1589931229-27 TIMED_WAITING tid=27 prio=5 IDLE | +> qtp1589931229-32 TIMED_WAITING tid=32 prio=5 IDLE | +> qtp1589931229-33 RUNNABLE tid=33 prio=5 SELECTING | +> qtp1589931229-28 TIMED_WAITING tid=28 prio=5 IDLE | +> qtp1589931229-30 TIMED_WAITING tid=30 prio=5 IDLE += ScheduledExecutorScheduler@1e20e272{STARTED} - STARTED += ServerConnector@70239ac0{HTTP/1.1, (http/1.1)}{0.0.0.0:9096} - STARTED | +~ Server@4db85c57{STARTING}[9.4.43.v20210629] - STARTING | +~ QueuedThreadPool[qtp1589931229]@5ec46cdd{STARTED,8<=8<=200,i=6,r=-1,q=0}[ReservedThreadExecutor@16627f86{s=0/2,p=0}]
• STARTED | +~ ScheduledExecutorScheduler@1e20e272{STARTED} - STARTED | +- org.eclipse.jetty.io.ArrayByteBufferPool@3a1e66f1 | += HttpConnectionFactory@dc1f9ce[HTTP/1.1] - STARTED | | +- HttpConfiguration@1960af83{32768/8192,8192/8192,https://:0,[]} | |
  +> customizers size=0 | | +> formEncodedMethods size=2 | | | +> POST | | | +> PUT | | +> outputBufferSize=32768 | | +> outputAggregationSize=8192 | | +> requestHeaderSize=8192 | | +> responseHeaderSize=8192 | | +> headerCacheSize=1024 | | +> secureScheme=https | | +> securePort=0 | | +> idleTimeout=-1 | | +> blockingTimeout=-1 | | +> sendDateHeader=true | | +> sendServerVersion=true | | +> sendXPoweredBy=false | | +> delayDispatchUntilContent=true | |
  +> persistentConnectionsEnabled=true | | +> maxErrorDispatches=10 | | +> minRequestDataRate=0 | | +> minResponseDataRate=0 | | +> cookieCompliance=RFC6265 | | +> setRequestCookieCompliance=RFC6265 | | +> notifyRemoteAsyncErrors=true | | +> relativeRedirectAllowed=false | += SelectorManager@ServerConnector@70239ac0{HTTP/1.1, (http/1.1)}{0.0.0.0:9096} - STARTED | | += ManagedSelector@6ade7db{STARTED} id=0 keys=0 selected=0 updates=0 - STARTED | | += EatWhatYouKill@6c4eab86/SelectorProducer@550ad4bd/PRODUCING/p=false/QueuedThreadPool[qtp1589931229]@5ec46cdd{STARTED,8<=8<=200,i=6,r=-1,q=0}[ReservedThreadExecutor@16627f86{s=0/2,p=0}][pc=0,pic=0,pec=0,epc=0]@2021-10-16T13:56:48.6782212+05:30
• STARTED | | | +- SelectorProducer@550ad4bd | | | +~ QueuedThreadPool[qtp1589931229]@5ec46cdd{STARTED,8<=8<=200,i=6,r=-1,q=0}[ReservedThreadExecutor@16627f86{s=0/2,p=0}]
• STARTED | | +> updates @ 2021-10-16T13:56:48.6732217+05:30 size=0 | | +> keys @ 2021-10-16T13:56:48.674221+05:30 size=0 | +- sun.nio.ch.ServerSocketChannelImpl[/0:0:0:0:0:0:0:0:9096] | +- qtp1589931229-31-acceptor-0@24c38c77-ServerConnector@70239ac0{HTTP/1.1, (http/1.1)}{0.0.0.0:9096} += HandlerList@34d20482{STARTED} - STARTED | += ResourceHandler@28a68c0{STARTED} - STARTED += ErrorHandler@61ad62a3{STARTED} - STARTED +> jdk.internal.loader.ClassLoaders$AppClassLoader@311d617d +> jdk.internal.loader.ClassLoaders$PlatformClassLoader@5e955596 key: +- bean, += managed, +~ unmanaged, +? auto, +: iterable, +] array, +@ map, +> undefined 2021-10-16 13:56:48.679:INFO:oejs.Server:Jetty Web Server Start Thread: Started @80567ms

I have been struggling on this from past few days, Can anyone please help me out?

Answer 1

Some changes I would recommend.

First, the ResourceHandler is doing nothing for you.

<Array type="org.eclipse.jetty.server.Handler">
  <Item>
    <New class="org.eclipse.jetty.server.handler.ResourceHandler">
      <Set name="directoriesListed">true</Set>
      <Set name="resourceBase">.</Set>
    </New>
  </Item>
</Array>

At best it sits there and does nothing. (as it has no context so no requests can reach it). At worst, the way you have set it up you've now shared your entire drive via that ResourceHandler.

Remove it.

Next, the declaration of your ServletContext ...

<New class="org.eclipse.jetty.servlet.ServletContextHandler">
  <Arg>
    <Ref id="Contexts"/>
  </Arg>
  <Arg>/cts/output</Arg>
  <Set name="resourceBase"><SystemProperty name="jetty.home" default="."/>/../cache/</Set>
  <Call name="addServlet">
    <Arg>com.myservlet.webservices.remote.FileProxyServlet</Arg>
    <Arg>/</Arg>
  </Call>

Things about this declaration ...

• It doesn't end it </New> so it's a broken XML.
  • Fix your XML.
• The first <Arg><Ref id="Contexts"/></Arg> is pointless and mangled as well. The first argument appears to be attempting to point to a Contexts reference somewhere else, but that reference doesn't exist anywhere, and your use of <Ref> element is wrong as well, you've declared the <Ref> itself with the id="Contexts" and are not referencing some other id, which would be <Ref refid="otherId"/>.
  • Remove that <Arg> entirely, it serves no purpose
• The use of <Arg>/cts/output</Arg> to set the context-path is wrong in multiple ways.
  • Context Path rules are (Must start with / and cannot contain any other slash anywhere else, no nested context paths, that's what url-patterns are for)
  • Don't attempt to use the constructor to set the context-path, use the setter.
  • Remove this <Arg> as well, declare a valid context path, and then use the setter instead <Set name="contextPath">/foo</Set>
• You add a servlet to / url-pattern, making the effort on resourceBase invalid.
  • Fix this url-pattern.
• Your ServletContextHandler is not part of the server handler tree as well. I think you were trying to do that with the first <Arg>, but that's not how you add it to the handler tree.
  • If you are declaring it directly, just put the <New class="ServletContextHandler">... in the handler tree itself.
  • If you are loading / instantiating the ServletContextHandler later, then you have to use the DeploymentManager and WebAppProvider along with an existing ContextHandlerCollection that is present in the handler tree already for the deployment to add it to.

You have a weird and frankly broken mix of usages that not even Jetty 6 should have allowed.

You cannot have FileProxyServlet on the default url-pattern (which is /) AND have static resources served at the same time.

Suggestion, use Jetty properly (these suggestions also work in Jetty 6 and all the way back to Jetty 4 no less!)

1. Don't mix ServletContextHandler and ResourceHandler.
2. Use the ServletContextHandler only.
3. Make the ServletContextHandler contexPath only /ctx
4. Make the ServletContextHandler resourceBase point to the location (directory or jar:file:// url) where your static file content is.
5. Add the normal DefaultServlet to your ServletContextHandler (make sure it's named "default" and is the only servlet mapped to /).
6. Add your com.myservlet.webservices.remote.FileProxyServlet to the url-pattern /output.
7. Finally, add your ServletContextHandler to the server Handler tree.

Here's some more information from past answers that I encourage you to read.

• Jetty: default servlet context path
• Serving static files from alternate path in embedded Jetty
• What is difference between ServletContextHandler.setResourceBase and ResourceHandler.setResourceBase when using Jetty embedded container?
• Embedded Jetty handling urls to serve content
• How to configure Jetty Handlers?
• Serving static files from alternate path in embedded Jetty
• Jetty: How to nest HandlerWrapper, HandlerList, and ContextHandlerCollection, and ContextHandler

Thing is, doing this entirely in a single XML file is just baffling (the use of a single XML file to configure Jetty was discouraged back when Jetty 6.0.0 was first released). Have you considered using the XML files that come with jetty-home already and just referencing them (in the correct order) in your own embedded-jetty startup?

That means you only have to define, by hand, the ServletContextHandler, and put it wherever you want your WebAppProvider to look for contexts to deploy.

Bonus is that your effort to upgrade Jetty versions becomes trivial.