2

I have services (both writtenin scala) deployed in kubernetes cluster, lets name them as external service and internal service

  1. External service - 5 pods, expose REST API, consume internal service grpc API thru grpc client
  2. Internal service 1 pod, expose grpc API

Problem - When I call internal service grpc API by port-forwarding using some grpc client e.g. BloomRPC, latency is very good within 1 sec (avg. 300 ms) but when I consume same grpc API using grpc client (written in scala), it always takes on average > 2 seconds. I understand first call will take time due to TPC connection handling but it always takes 2-4 seconds even after JVM warm-up. I am creating grpc channel only once at start of service (in main method, asynchronous stub) and reusing the same, no client side grpc load balancing enabled. How to improve latency?

class MyServiceGrpcClient(val channel: ManagedChannel) {
  val myServiceStub = MyServiceGrpc.stub(channel)

  def this(host: String, port: Int) {
    this(ManagedChannelBuilder.forAddress(host, port).usePlaintext().build())
  }

  @throws[InterruptedException]
  def shutdown(): Unit = {
    channel.shutdown.awaitTermination(5, TimeUnit.SECONDS)
  }
}


object ExternalServiceApp extends App {
  val host = sys.env.getOrElse("HOST_NAME", "internal-service-name") //Its k8s ClusterIP service name
  val port = sys.env.getOrElse("PORT", "1234")
  val internalGrpcClient = new MyServiceGrpcClient(wesHost, wesPort).myServiceStub

  val routes = new ExternalRoutes(internalGrpcClient)

  //Akka http routes logic, route API calls grpc API
}

class ExternalRoutes(grpcClient: MyServiceGrpcClient){

  override def getResponse(request: Request): Future[Response] = {

    grpcClient.internalGrpcApi(request).map { resp =>
      resp
    }

  }
}
Abhay
  • 928
  • 1
  • 10
  • 28
  • Can you post the entire client code? – thesamet Jan 15 '21 at 00:09
  • @thesamet Updated code with Main App class where external Http API creates grpc client object and it passed to another class where internal grpc call happens. So stub object(with channel) only created once at the start of external service – Abhay Jan 15 '21 at 02:38
  • I suggest trying to do a divide and conquer to identify which part of the system is causing the problem. Can you try grpc-java directly without ScalaPB? Can you try connecting to a locally running server so we know it's not the network or Kubernetes? Keep going like this until you know which part is adding the latency. – thesamet Jan 15 '21 at 15:17
  • I tried below things 1.Port forward internal grpc service and hit grpc API using bloomrpc grpc client. It gives real good latency within (1 sec) 2.Run external http service locally and port forward internal grpc service and hit external http API (in code it calls grpc api using above written grpc client code), then this line takes > 2 sec grpcClient.internalGrpcApi(request).map { resp => resp } You want me to generate client classes in java using protoc compiler and try calling grpc API? Do you think scalapb classes taking more time? – Abhay Jan 15 '21 at 15:58
  • ScalaPB has similar performance to grpc-java.It's worth verifying that this is true for your use case as well, so we rule out ScalaPB as the culprit, or find that it is and zoom in on it. – thesamet Jan 16 '21 at 16:27
  • Can you have an experiment that does not involve port forwarding? Simply run a client and server on the same host and connect directly? – thesamet Jan 16 '21 at 16:31
  • @thesamet I ran server and client locally on single machine. It works perfectly fine. I can get response in < 200 ms even with large payload size. Any idea what could be the issue? – Abhay Feb 03 '21 at 20:27
  • It's hard to tell what could cause this based on the information you provided and without being able to access your system. – thesamet Feb 04 '21 at 21:17

0 Answers0