gRPC future callback got exception:StatusRuntimeException: CANCELLED

Question

io.grpc.StatusRuntimeException: CANCELLED
    at io.grpc.Status.asRuntimeException(Status.java:539)
    at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:439)
    at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:422)
    at io.grpc.internal.ClientCallImpl.access$100(ClientCallImpl.java:74)
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:508)
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$700(ClientCallImpl.java:425)
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:540)
    at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:52)
    at io.grpc.internal.SerializeReentrantCallsDirectExecutor.execute(SerializeReentrantCallsDirectExecutor.java:64)
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.closed(ClientCallImpl.java:544)
    at io.grpc.internal.AbstractClientStream2$TransportState.closeListener(AbstractClientStream2.java:307)
    at io.grpc.internal.AbstractClientStream2$TransportState.transportReportStatus(AbstractClientStream2.java:287)
    at io.grpc.netty.NettyClientHandler.cancelStream(NettyClientHandler.java:455)
    at io.grpc.netty.NettyClientHandler.write(NettyClientHandler.java:231)
    at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:739)
    at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:731)
    at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:817)
    at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:724)
    at io.netty.channel.DefaultChannelPipeline.write(DefaultChannelPipeline.java:1022)
    at io.netty.channel.AbstractChannel.write(AbstractChannel.java:291)
    at io.grpc.netty.WriteQueue.flush(WriteQueue.java:124)
    at io.grpc.netty.WriteQueue.access$000(WriteQueue.java:46)
    at io.grpc.netty.WriteQueue$1.run(WriteQueue.java:58)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:445)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
    at java.lang.Thread.run(Thread.java:745)

I got an exception as above when i call another grpc service in a grpc server handle process with future & callback.

My code like this:

handle(){

....

***FutureClient futureClient = ...
ListenableFuture<***> future = futureClient.call(...);
Futures.addCallback(future, new FutureCallback<>() {...});

...

}

The access was canceled when the process handle() completed.

How can I resolve the problem?

Cancel happens when there is a client side error, or it was cancelled by application code (i.e. your code). Are you sure you aren't cancelling the future somewhere? — Carl Mastrangelo, May 09 '17 at 23:20

score 14 · Answer 1 · answered May 10 '17 at 16:22

14

Outgoing RPCs from a server are implicitly tied to the incoming request that triggered them, via io.grpc.Context. If you are doing an RPC that proceeds even after the incoming RPC completed you should use context.fork():

Context forked = Context.current().fork();
Context old = forked.attach();
try {
  // RPCs at this point can continue after the incoming RPC completes
  futureClient.call(...);
} finally {
  forked.detach(old);
}

Context.run(Runnable) is also a convenient alternative to using attach/detach directly.

answered May 10 '17 at 16:22

Eric Anderson

24,057
5
55
76

1

This was exactly what I was looking for! I hope this would be clearly stated also in documentation. – Kitanotori May 22 '18 at 05:43
@Eric Anderson Could you please elaborate on what "proceeds even after the incoming RPC completed" ? I too encountered the issue. Client makes a unary request, and server sends out a stream of responses in the same thread. So, I fail to see how the "incoming RPC is completed"? Does it have more to do with timeout or something? – Uma Priyadarsi Jan 28 '22 at 08:35
1

@UmaPriyadarsi, the case here is when a server creates an RPC to process an RPC (that is, the server is also a client). By default that "child" RPC cannot outlive the parent RPC. In your case it sounds like the server just responds; there's no second RPC involved. – Eric Anderson Feb 03 '22 at 17:37
@EricAnderson yeah I got my doubt resolved over here : https://github.com/grpc/grpc-java/issues/8876#event-5990360727 – Uma Priyadarsi Feb 04 '22 at 07:36
HI @EricAnderson what's the difference between your approach here and using `ServerCallStreamObserver` described here: https://stackoverflow.com/questions/54588382/how-can-a-grpc-server-notice-that-the-client-has-cancelled-a-server-side-streami – macemers May 15 '23 at 02:55
@macemers, "what do you want to happen if the client cancels the RPC?" If you want any outgoing RPCs to be cancelled and also to stop processing, then you'd follow that other answer. If you want outgoing RPCs to continue and to continue processing, you'd use this answer. – Eric Anderson May 16 '23 at 15:11

gRPC future callback got exception:StatusRuntimeException: CANCELLED

1 Answers1

Linked