I am just starting with OpenTelemetry and have created two (micro)services for this purpose: Standard and GeoMap.
The end-user sends requests to the Standard service, who in turn sends requests to GeoMap to fetch informations before returning the result to the end-user. I am using gRPC for all communications.
I have instrumented my functions as such:
For Standard:
func (s *server) GetStandard(ctx context.Context, in *pb.GetStandardRequest) (*pb.GetStandardResponse, error) {
newCtx, span := otel.Tracer(name).Start(ctx, "GetStandard")
span.SpanContext()
defer span.End()
countryInfo, err := geoMapServiceClient.GetCountry(newCtx,
&pb.GetCountryRequest{
Name: in.Name,
})
....
return &pb.GetStandardResponse{
Standard: standard,
}, nil
}
For GeoMap
func (s *geomapService) GetCountry(ctx context.Context, in *pb.GetCountryRequest) (*pb.GetCountryResponse, error) {
_, span := otel.Tracer(name).Start(ctx, "GetCountry")
defer span.End()
span.SetAttributes(attribute.String("country", in.Name))
...
return &pb.GetCountryResponse{
Country: &country,
}, nil
}
Both services are configured to send their traces to a Jaeger Backend:
func tracerProvider(url string) (*tracesdk.TracerProvider, error) {
// Create the Jaeger exporter
exp, err := jaeger.New(jaeger.WithCollectorEndpoint(jaeger.WithEndpoint(url)))
if err != nil {
return nil, err
}
tp := tracesdk.NewTracerProvider(
tracesdk.WithBatcher(exp),
tracesdk.WithResource(resource.NewWithAttributes(
semconv.SchemaURL,
semconv.ServiceName(service),
attribute.String("environment", environment),
attribute.Int64("ID", id),
)),
)
return tp, nil
}
func main() {
tp, err := tracerProvider("http://localhost:14268/api/traces")
if err != nil {
log.Fatal(err)
}
defer func() {
if err := tp.Shutdown(context.Background()); err != nil {
log.Fatal(err)
}
}()
otel.SetTracerProvider(tp)
...
}
I would expect the span generated in the GeoMap service to be a child span of the Standard service (since it is the Standard service who is making the request through the geoMapServiceClient). However these spans are totally unrelated in Jaeger:
What am I missing here?
EDIT:
I have managed to include interceptors as @Peter suggested in the comments. Now spans are linked to each other and I can see that my parent span from the StandardService is making a request to the GeoMapService:
However, I have defined an attribute in my function call (see the span.SetAttributes(attribute.String("country", in.Name))) but I am unable to see that value from within my child span.
For that I have to switch to the other trace generated by the service GeoMap:
I don't need to have two different traces actually, I would like to have everything in the same trace (one end-user query should result in a single trace). How should I change my code from here?
