2

I'm building a GRPC server in GO and I want increase its performance. I've increased my machine and put 2CPUS in order to make it better but I noticed that my server doesn't use all cpu cores and I couldn't find a way to fix it (I've run some tests to make the server worker harder). It seems like my server uses 1 cpu a lot and the other not. My load average stay above 1 what means that my server is full of requests but when I see the performance in netdata there's only one CPU working.

This is my grpc code:

package main
import (
    "fmt"
    "runtime"
    "log"
    "net"
    "google.golang.org/grpc"
    "golang.org/x/net/context"
    "google.golang.org/grpc/reflection"
    pb "teste/prototeste"

)
func (s *server) Test(ctx context.Context, n *pb.TestRequest)      (*pb.TestReply, error){
    return &pb.TestReply{Message: n.Name}, nil
}

type server struct{}

const (
    port = ":50051"
)

func main(){

    numOfCores := runtime.NumCPU()
    fmt.Println(numOfCores)
    runtime.GOMAXPROCS(numOfCores)
    lis, err := net.Listen("tcp", port)
    if err != nil {
        log.Fatalf("failed to listen: %v", err)
    }
    s := grpc.NewServer()
    pb.RegisterGreeterServer(s, &server{})
    // Register reflection service on gRPC server.
    reflection.Register(s)

    s.Serve(lis)

}

Notice that in Serve function there's a goroutine that allow the server accept multiple requests in parallel.

API reference grpc in go

This is my cpu usage

Any tips to how to solve this and make my server use 2 cpus?

Vivi
  • 693
  • 2
  • 11
  • 21
  • Are you using Docker? A virtual machine? Launching it through a systemd unit with CPUAffinity set? Maybe your server has CPUAffinity set to 1 or 2 CPUs in the user.slice. Maybe your OS kernel is using some kind of setting that drives network traffic to a CPU per connection hash and you're only testing from one IP. . . – Zan Lynx Mar 07 '17 at 21:09
  • I run many tests with 1 and 2 differents IPs and the result was the same. I'm using a virtual machine. What is this CPUAffinity? I didn't know about this. – Vivi Mar 07 '17 at 21:13
  • Ah, and the server is running in a linux centos – Vivi Mar 07 '17 at 21:15
  • If you're running in a virtual machine, how many CPU's did you configure the virtual machine to use? – Zan Lynx Mar 07 '17 at 21:17
  • Some information about packet steering which might apply https://www.kernel.org/doc/Documentation/networking/scaling.txt – Zan Lynx Mar 07 '17 at 21:18
  • I've configured 2 CPUs and the result was the same as 1 CPU – Vivi Mar 07 '17 at 21:20
  • I saw my process affinity and it gives me: "pid 31255's current affinity list: 0,1" What means it can run in one of my 2 CPUs – Vivi Mar 07 '17 at 21:25

1 Answers1

1

In gRPC, each server handler runs in its own goroutine. From there it is up to the Go scheduler to determine what CPUs to schedule them on. Depending on what your server handlers are doing, system load, etc, it's possible they could all end up scheduled on the same CPU.

Doug Fawley
  • 953
  • 5
  • 7