7

Play 2.5.0 uses Netty 4.0.33, while gRPC requires Netty 4.1.0 (for http2 support), which causes the following exception:

[error] p.c.s.n.PlayRequestHandler - Exception caught in Netty
java.lang.AbstractMethodError: null
    at io.netty.util.ReferenceCountUtil.touch(ReferenceCountUtil.java:73)
    at io.netty.channel.DefaultChannelPipeline.touch(DefaultChannelPipeline.java:84)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:154)
    at com.typesafe.netty.http.HttpStreamsHandler.channelRead(HttpStreamsHandler.java:131)
    at com.typesafe.netty.http.HttpStreamsServerHandler.channelRead(HttpStreamsServerHandler.java:96)
    at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
    at io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:154)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:154)
    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
    at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
[error] p.c.s.n.PlayRequestHandler - Exception caught in Netty
java.util.NoSuchElementException: http-handler-body-publisher
    at io.netty.channel.DefaultChannelPipeline.getContextOrDie(DefaultChannelPipeline.java:1050)
    at io.netty.channel.DefaultChannelPipeline.remove(DefaultChannelPipeline.java:379)
    at com.typesafe.netty.http.HttpStreamsHandler.handleReadHttpContent(HttpStreamsHandler.java:191)
    at com.typesafe.netty.http.HttpStreamsHandler.channelRead(HttpStreamsHandler.java:167)
    at com.typesafe.netty.http.HttpStreamsServerHandler.channelRead(HttpStreamsServerHandler.java:96)
    at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
    at io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:154)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:154)
    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
    at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)

After I remove all gRPC codes, it works again.

Is there any quick fix that I can try now? Thanks!

DANG Fan
  • 854
  • 11
  • 21
  • Netty 4.0.33 and Netty 4.1.0 are not binary compatible. You can try to [force](http://www.scala-sbt.org/0.13/docs/Library-Management.html#Forcing+a+revision+%28Not+recommended%29) the 4.0.33 revision (which is not recommended), but I doubt it will work. Looks like you will need a alternative to gRPC. – marcospereira Mar 06 '16 at 21:00
  • @marcospereira force 4.0.33 won't work due to gRPC requires http2 which is a new feature in netty 4.1.0. I tried to compile Play 2.5 using netty 4.1.0, but it failed. As a result, I downgrade to Play 2.4 for now. – DANG Fan Mar 07 '16 at 04:04
  • Does it works with play 2.4? – marcospereira Mar 07 '16 at 12:13
  • @marcospereira Yes, it does. – DANG Fan Mar 07 '16 at 12:27
  • I'm having the same problem but with a different library: Redisson v3.5.7. I'm on Scala 2.11 and sbt 0.13. Probably not much I can do about it. – Nader Ghanbari Jan 04 '18 at 01:20
  • I'm having the same problem when added Firebase Admin 6.12.2 with scala 2.11 and play 2.5.4 – Rajat Feb 27 '20 at 07:09

3 Answers3

12

Edit:

Play 2.6.0 was released, and it is using Netty 4.1.

tl;dr

Play 2.5.0 and gRPC are not compatible due to binary incompatibilities between Netty 4 and Netty 4.1.

Here is why it doesn't work with Play 2.5.0 but works with Play 2.4.0:

Play 2.5.0 uses Netty version 4.0.33.Final, which is declared (transitively via netty-reactive-streams version 1.0.2) like this:

<dependency>
    <groupId>io.netty</groupId>
    <artifactId>netty-handler</artifactId>
    <version>4.0.33.Final</version>
</dependency>
<dependency>
    <groupId>io.netty</groupId>
    <artifactId>netty-codec-http</artifactId>
    <version>4.0.33.Final</version>
</dependency>

Version 4.0.33.Final is evicted by version 4.0.34.Final which was added transitively by async-http-client and these are the netty dependencies used by Play 2.5.0:

io.netty:netty-buffer:4.0.34.Final
io.netty:netty-codec-http:4.0.34.Final
io.netty:netty-codec:4.0.34.Final
io.netty:netty-common:4.0.34.Final
io.netty:netty-handler:4.0.34.Final
io.netty:netty-transport:4.0.34.Final

Play 2.4.6, on the other side, requires just the following netty dependency:

io.netty:netty:3.8.0.Final

So, while Play 2.5.0 depends on a lot of smaller netty packages, Play 2.4.6 depends just on a single netty package. That is because Netty 4 changed the project structure to split the project into multiple subprojects so that a user can add just the necessary features from Netty. And even more important, "the package name of Netty has been changed from org.jboss.netty to io.netty".

Why are these changes both relevant here?

  1. There is no dependency conflict between gRPC with 2.4.6 because gRPC does not require dependency io.netty:netty, not even transitively. Because of that, there is no eviction.
  2. There is no conflict at the class level because the classes have different packages names (org.jboss.netty and io.netty) and then distinct full qualified names. Therefore, they are treated as different classes.

There are conflicts with Play 2.5.0 because Netty 4 and Netty 4.1 have both the same dependencies artifactId (then 4.1 version evicts 4.0.34) and because - since we now have the same full qualified class names between these two versions - binaries incompatibilities arise.

That is a long explanation to say that there is no way to make gRPC and Play 2.5.0 work together right now with Netty. Even if you decide to use Akka HTTP server backend, there is a chance of conflict with Play WS.

marcospereira
  • 12,045
  • 3
  • 46
  • 52
  • Isnt it possible to solve this using different classloaders for play and for grpc, so that each loads whatever they need... – vach Apr 29 '16 at 08:30
  • So essentially you would let play load whatever it neads and then create a classloader for grpc and make sure any thread running grpc stuff uses grpc classloader... probably will be ugly – vach Apr 29 '16 at 08:46
  • Good luck doing that, specially considering both development and production environments. – marcospereira Apr 29 '16 at 13:15
  • I need your opinion on this one, i'm considering to do this. First what may be problem with dev / prod environments? if it works on dev why would it be any different for prod? Second the solution i think of is create an Aspect which will wrap any grpc instantiation or execution with around aspect which will set the appropriate classloader before and set the orignal one after the execution... do you think that will work? – vach May 03 '16 at 04:35
  • The problem between dev and prod envs is that they have different ways to load code (per instance, dev env does hot reload of code). I wouldn't be surprised if after digging you discover that both environments are using specific classloaders. About using an aspect, if you set another class loader while running grpc code, what will happen with the code that is running in parallel? I really don't know if this will works (maybe since jrebel exists?), but pay a lot of attention to avoid leaks. Anyway, I really don't think it is worth the effort. What are the alternatives to grpc here? – marcospereira May 03 '16 at 06:10
  • apache trift but as far as i can see it sucks, besides i believe this will be resolved pretty quickly, play devs should be interested to fix this, after all proto is the best for what it is doing... – vach May 04 '16 at 03:32
  • What about using websockets instead of grpc? – marcospereira May 04 '16 at 03:34
  • Websockets make sense if you're communicating with Browser, in all other ways grpc is the best cos you have all 4 possible ways of communication... request -> response (like regular rest), request -> stream, stream -> response and stream -> stream (like websocket)... – vach May 04 '16 at 03:36
  • This is how i understand class loaders, you can have 2 classloaders loaded with 2 different versions of classes, and even in single thread you'll be able to switch classloaders in runtime, in order to make it all work you need to make sure that when the classes are used to instantiate objects you are on the right classloader context... correct me if i'm wrong... Once you instantiated them with different versions, you should be safe using common interface or methods that exist in both versions... but yeah this is pain :) – vach May 04 '16 at 03:38
  • 1
    @vach see my last edit here. Play 2.6.0 will be release with Netty 4.1. – marcospereira Mar 13 '17 at 13:55
0

I have also come across this in Play using Java. Are you using a gRPC client or server interface with the Play server? If you are just using just a client you can overcome this by using the OKHTTP client version instead of one that relies on Netty.

https://mvnrepository.com/artifact/io.grpc/grpc-okhttp

If you are using a server, I think you might be out of luck. You can try to exclude Netty from gRPC and hope Play's version is sufficient. Just add excludeAll ExclusionRule(organization = "io.netty") to the gRPC import.

Here is a thread on the play mailing list on the topic, but they do not have any changes on the topic for now: https://groups.google.com/forum/#!topic/play-framework/TWa18IfZ5kA

Ben
  • 2,314
  • 1
  • 19
  • 36
0

You can also use the maven-shade-plugin to relocate the the netty package names to not clash with the earlier version.

Have a look at couchbase jvm core library for an example https://github.com/couchbase/couchbase-jvm-core.

deephacks
  • 151
  • 9