1

I am using akka grpc ( v 10.2.0 ),

I am creating 100 clients in parallel ( stress testing the grpc server ). From time to time, on client side I see errors like:

o.grpc.netty.shaded.io.netty.channel.unix.Errors$NativeIoException: writevAddresses(..) failed: Broken pipe
io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
    at io.grpc.Status.asRuntimeException(Status.java:533)
    at akka.grpc.internal.AkkaNettyGrpcClientGraphStage$$anon$1.onCallClosed(AkkaNettyGrpcClientGraphStage.scala:165)
    at akka.grpc.internal.AkkaNettyGrpcClientGraphStage$$anon$1.$anonfun$callback$1(AkkaNettyGrpcClientGraphStage.scala:71)
    at akka.grpc.internal.AkkaNettyGrpcClientGraphStage$$anon$1.$anonfun$callback$1$adapted(AkkaNettyGrpcClientGraphStage.scala:67)
    at akka.stream.impl.fusing.GraphInterpreter.runAsyncInput(GraphInterpreter.scala:466)
    at akka.stream.impl.fusing.GraphInterpreterShell$AsyncInput.execute(ActorGraphInterpreter.scala:498)
    at akka.stream.impl.fusing.GraphInterpreterShell.processEvent(ActorGraphInterpreter.scala:600)
    at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:769)
    at akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:784)
    at akka.actor.Actor.aroundReceive(Actor.scala:537)
    at akka.actor.Actor.aroundReceive$(Actor.scala:535)
    at akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:691)
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:577)
    at akka.actor.ActorCell.invoke(ActorCell.scala:547)
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)
    at akka.dispatch.Mailbox.run(Mailbox.scala:231)
    at akka.dispatch.Mailbox.exec(Mailbox.scala:243)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: io.grpc.netty.shaded.io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer

but on server all is fine, no errors. Sometimes this client ( which generated 100 connections in parallel ) is just hanging. Stream hangs no errors.

If I will restart client, all will start working fine, so server is ok. Not sure where to look for an issue ?

FrancMo
  • 2,459
  • 2
  • 19
  • 39
  • 2
    I think this is related to https://github.com/googleapis/google-cloud-java/issues/2502 and I would test using deadlines as it is shown here https://grpc.io/blog/deadlines/ – Felipe Jan 21 '21 at 15:27
  • I am having a similar issue with GRPC for a long running service. I am testing out the deadlines as you have pointed out. – Archimedes Trajano Apr 05 '21 at 20:14

0 Answers0