Why am i getting connection connection closed before server preface received in go?

Question

I am trying to setup a rpc server and a proxy HTTP server over GRPC server on same port using grpc-gateway. Wierdly some times i am getting failed to receive server preface within timeout error randomly. Most of the times it happens on service restarts. It starts working and returns proper response after couple of retries. I am not sure what's happening. Can somebody help me out ? Here is the service startup snippet

func makeHttpServer(conn *grpc.ClientConn) *runtime.ServeMux {
    router := runtime.NewServeMux()

    if err := services.RegisterHealthServiceHandler(context.Background(), router, conn); err != nil {
        log.Logger.Error("Failed to register gateway", zap.Error(err))
    nricher
    if err := services.RegisterConstraintsServiceHandler(context.Background(), router, conn); err != nil {
        log.Logger.Error("Failed to register gateway", zap.Error(err))
    }
    return router
}

func makeGrpcServer(address string) (*grpc.ClientConn, *grpc.Server) {

    grpcServer := grpc.NewServer()
    services.RegisterHealthServiceServer(grpcServer, health.Svc{})
    services.RegisterABCServer(grpcServer, ABC.Svc{})
    conn, err := grpc.DialContext(
        context.Background(),
        address,
        grpc.WithInsecure(),
    )
    if err != nil {
        log.Logger.Error("Failed to dial server", zap.Error(err))
    }

    return conn, grpcServer
}

func httpGrpcRouter(grpcServer *grpc.Server, httpHandler *runtime.ServeMux, listener net.Listener) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        if r.ProtoMajor == 2 {
            grpcServer.Serve(listener)
        } else {
            httpHandler.ServeHTTP(w, r)
        }
    })
}

func Start() error {
    conf := config.Get()
    address := fmt.Sprintf("%s:%d", conf.ServerHost, conf.ServerPort)

    listener, err := net.Listen("tcp", address)
    if err != nil {
        log.Logger.Fatal("failed to listen: %v", zap.Error(err))
    }
    conn, grpcServer := makeGrpcServer(address)
    router := makeHttpServer(conn)

    log.Logger.Info("Starting server on address : " + address)
    err = http.Serve(listener, httpGrpcRouter(grpcServer, router, listener))
    return err
}

I'm getting a similar error: `connection closed before server preface received` when I test without TLS. If I add TLS everything works as expected. Ideally I would not want to manage certificates for my grpc micro services and just terminate TLS at load balancer — skud, Apr 18 '22 at 14:45
not necessarily the same issue, since this question is about grpc-gateway, but check also this: https://stackoverflow.com/questions/71627962/grpc-error-how-to-solve-connection-closed-before-server-preface-received — blackgreen, Oct 10 '22 at 12:56

score 2 · Answer 1 · answered Nov 18 '21 at 13:02

2

Try wrapping your router with h2c.NewHandler so the the http.Serve() call looks as follows:

err = http.Serve(listener, h2c.NewHandler(
        httpGrpcRouter(grpcServer, router, listener),
        &http2.Server{})
)

answered Nov 18 '21 at 13:02

jyshin

841
1
8
15

Thank you ser. This solved my problem. I changed my `httpSrv.Handler = grpcSrv` to `httpSrv.Handler = h2c.NewHandler(grpcSrv, &http2.Server{})`. Is this happening because there are issues supporting http2 without TLS? – skud Apr 18 '22 at 15:00
From my understanding, gRPC works on HTTP/2. So if you want to serve both HTTP/1 and gRPC on the same port, you need to handle h2c request to upgrade http/1 connection to http/2 and serve gRPC with the http/2 connection. – jyshin Apr 25 '22 at 06:04

Why am i getting connection connection closed before server preface received in go?

1 Answers1