occasional EOFException from camel (jetty)

Question

We have an application that using Apache camel to call REST API. And we got EOFExceptions from jetty occasionally.

Some more observations:

we're not using https
it happens very sporadically (once for per 10K ?)

Does anybody known the reason and how to resolve this problem?

Thaks.

Here's the stack trace

org.apache.camel.CamelExchangeException: JettyClient failed cause by: HttpConnectionOverHTTP@413b2b34::SocketChannelEndPoint@3af7292{localhost/127.0.0.1:44904<->/127.0.0.1:56676,ISHUT,fill=-,flush=-,to=0/0}{io=1/0,kio=1,kro=1}->HttpConnectionOverHTTP@413b2b34(l:/127.0.0.1:56676 <-> r:localhost/127.0.0.1:44904,closed=false)=>HttpChannelOverHTTP@1f576404(exchange=HttpExchange@56d8659f req=TERMINATED/null@null res=PENDING/null@null)[send=HttpSenderOverHTTP@37f033df(req=QUEUED,snd=COMPLETED,failure=null)[HttpGenerator@2f8acc9e{s=START}],recv=HttpReceiverOverHTTP@5749001f(rsp=IDLE,failure=null)[HttpParser{s=CLOSED,0 of -1}]]. Exchange[ID-SPAMDBP1-1641410557693-0-168236]. Caused by: [java.io.EOFException - HttpConnectionOverHTTP@413b2b34::SocketChannelEndPoint@3af7292{localhost/127.0.0.1:44904<->/127.0.0.1:56676,ISHUT,fill=-,flush=-,to=0/0}{io=1/0,kio=1,kro=1}->HttpConnectionOverHTTP@413b2b34(l:/127.0.0.1:56676 <-> r:localhost/127.0.0.1:44904,closed=false)=>HttpChannelOverHTTP@1f576404(exchange=HttpExchange@56d8659f req=TERMINATED/null@null res=PENDING/null@null)[send=HttpSenderOverHTTP@37f033df(req=QUEUED,snd=COMPLETED,failure=null)[HttpGenerator@2f8acc9e{s=START}],recv=HttpReceiverOverHTTP@5749001f(rsp=IDLE,failure=null)[HttpParser{s=CLOSED,0 of -1}]]]
    at org.apache.camel.component.jetty9.JettyContentExchange9.doTaskCompleted(JettyContentExchange9.java:162)
    at org.apache.camel.component.jetty9.JettyContentExchange9$2.onComplete(JettyContentExchange9.java:246)
    at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:202)
    at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:194)
    at org.eclipse.jetty.client.HttpReceiver.terminateResponse(HttpReceiver.java:470)
    at org.eclipse.jetty.client.HttpReceiver.abort(HttpReceiver.java:552)
    at org.eclipse.jetty.client.HttpReceiver.responseFailure(HttpReceiver.java:442)
    at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.failAndClose(HttpReceiverOverHTTP.java:369)
    at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.earlyEOF(HttpReceiverOverHTTP.java:338)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1551)
    at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.shutdown(HttpReceiverOverHTTP.java:209)
    at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.process(HttpReceiverOverHTTP.java:147)
    at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.receive(HttpReceiverOverHTTP.java:73)
    at org.eclipse.jetty.client.http.HttpChannelOverHTTP.receive(HttpChannelOverHTTP.java:133)
    at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onFillable(HttpConnectionOverHTTP.java:155)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
    at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
    at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
    at java.lang.Thread.run(Unknown Source)
Caused by: java.io.EOFException: HttpConnectionOverHTTP@413b2b34::SocketChannelEndPoint@3af7292{localhost/127.0.0.1:44904<->/127.0.0.1:56676,ISHUT,fill=-,flush=-,to=0/0}{io=1/0,kio=1,kro=1}->HttpConnectionOverHTTP@413b2b34(l:/127.0.0.1:56676 <-> r:localhost/127.0.0.1:44904,closed=false)=>HttpChannelOverHTTP@1f576404(exchange=HttpExchange@56d8659f req=TERMINATED/null@null res=PENDING/null@null)[send=HttpSenderOverHTTP@37f033df(req=QUEUED,snd=COMPLETED,failure=null)[HttpGenerator@2f8acc9e{s=START}],recv=HttpReceiverOverHTTP@5749001f(rsp=IDLE,failure=null)[HttpParser{s=CLOSED,0 of -1}]]
    ... 18 common frames omitted

Just a hunch but the responses that get cut short are they large responses? I know Jetty has some pretty small defaults for request and response size. Maybe look into this. See this question https://stackoverflow.com/questions/49228950/jetty-and-max-content-size — Namphibian, Jan 21 '22 at 03:10
@Namphibian Thank you for your reply. I will check the size of the request and response. — lampmagic, Jan 29 '22 at 08:53
I couldn't find out the reason of the error. So I make it retry the request while catching an exception. — lampmagic, Jun 10 '22 at 09:15

occasional EOFException from camel (jetty)

0 Answers0