Adding Opentelemetry to .Net Core worker service

Question

I am trying to add opentelemetry to one of my .net core 5.0 worker service project everything works ok but the traces are not begin sent to the Jaeger UI, I also notice the ActivitySource.StartActivity return null, I searched on google and looks like I might be missing the ActivirtListner but not sure how to add to my worker service,

public static IHostBuilder CreateHostBuilder(string[] args) =>
    Host.CreateDefaultBuilder(args)
        .ConfigureServices((hostContext, services) =>
        {
            services.AddHostedService<Worker>();
            services.AddOpenTelemetryTracing(builder =>
            {
                //IConfiguration config = sp.GetRequiredService<IConfiguration>();
                builder
                    //.SetSampler(new AlwaysOnSampler())
                    .AddHttpClientInstrumentation()
                    //.AddSqlClientInstrumentation()
                    //.AddNServiceBusInstrumentation()
                    //.AddRabbitMqInstrumentation()
                    .AddSource(nameof(Worker))
                    .SetResourceBuilder(ResourceBuilder.CreateDefault().AddService("WorkerServiceForTelemetry"))
                    .AddConsoleExporter()
                    .AddAspNetCoreInstrumentation()
                    //.AddOtlpExporter(options => options.Endpoint = new Uri("http://localhost:4317"))
                    .AddJaegerExporter(opts =>
                    {
                        opts.AgentHost = "Jaeger";
                        opts.AgentPort = 6831;
                        opts.ExportProcessorType = ExportProcessorType.Simple;
                    });
            });
        }

here is my worker.cs file

using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using MessageType;
using PublishToRabbitMq;
using System.Diagnostics;
using OpenTelemetry.Context.Propagation;

namespace WorkerServiceForTelemetry
{
    public class Worker : BackgroundService
    {
        private readonly ILogger<Worker> _logger;
        private static readonly ActivitySource _activity = new(nameof(Worker));
        private static readonly TextMapPropagator Propagator = Propagators.DefaultTextMapPropagator;

        public Worker(ILogger<Worker> logger)
        {
            _logger = logger;
        }

        protected override async Task ExecuteAsync(CancellationToken stoppingToken)
        {
            //using (var sample = new InstrumentationWithActivitySource())
            //{
            //    sample.Start();
            //    System.Console.WriteLine("Traces are being created and exported" +
            //        "to Jaeger in the background. Use Jaeger to view them." +
            //        "Press ENTER to stop.");
            //    System.Console.ReadLine();
            //}

            while (!stoppingToken.IsCancellationRequested)
            {
                _logger.LogInformation("Worker running at: {time}", DateTimeOffset.Now);             
                DoWork();
                await Task.Delay(10000, stoppingToken);
            }
        }

        public void DoWork()
        {
            //using var listener = new ActivityListener
            //{
            //    ShouldListenTo = _ => true,
            //    Sample = (ref ActivityCreationOptions<ActivityContext> _) => ActivitySamplingResult.AllData,
            //    ActivityStarted = activity => Console.WriteLine($"{activity.ParentId}:{activity.Id} - Start"),
            //    ActivityStopped = activity => Console.WriteLine($"{activity.ParentId}:{activity.Id} - Stop")
            //};

            //ActivitySource.AddActivityListener(listener);

            //var parentContext = Propagator.Extract(default,
            //        ea.BasicProperties,
            //        ActivityHelper.ExtractTraceContextFromBasicProperties);

            //Baggage.Current = parentContext.Baggage;
            Console.WriteLine("this is WorkerServiceForTelemetry is running");
            Console.WriteLine("Publish Message");
            Publish publish = new Publish();
            var context = Activity.Current?.Context;
            for (var i=0; i<=1; i++)
            {
                using (var activity = _activity.StartActivity("RabbitMq Publish", ActivityKind.Server))
                {
                    PersonName person = new PersonName();
                    person.FirstName = "Zafar" + i;
                    person.LastName = "Jamal" + i;
                    publish.SendMessage(person);
                }
            }
        }
    }
}

This blog post may be able to help explain how to add activities to an app using Hosting. https://devblogs.microsoft.com/aspnet/observability-asp-net-core-apps/ — Muhammad Rehan Saeed, Apr 27 '21 at 10:47
I couldn't figure this out either, [logged an issue](https://github.com/open-telemetry/opentelemetry-dotnet/issues/2035) — djeikyb, May 05 '21 at 07:52
If you place a breakpoint inside the using block where you're starting the activity, you'll probably notice that StartActivity method is returning a null. This is because no listener got added to the ActivitySource. Replace ActivitySource.AddActivityListener(listener); with ActivitySource.AddActivityListener(new ActivityListener() {....}); — Akhilesh, Aug 04 '22 at 19:18
Don't forget that the AddJaegerExporter is about to be deprecated https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/src/OpenTelemetry.Exporter.Jaeger/README.md — Tore Nestenius, May 09 '23 at 09:27

score 2 · Answer 1 · answered Aug 04 '22 at 19:23

If you place a breakpoint inside the using block where you're starting the activity, you'll probably notice that StartActivity method is returning a null. This is because no listener got added to the ActivitySource.

Replace

ActivitySource.AddActivityListener(listener);

with

ActivitySource.AddActivityListener(new ActivityListener()
{
    ShouldListenTo = _ => true,
    Sample = (ref ActivityCreationOptions<ActivityContext> _) => ActivitySamplingResult.AllData,
    ActivityStarted = activity => Console.WriteLine($"{activity.ParentId}:{activity.Id} - Start"),
    ActivityStopped = activity => Console.WriteLine($"{activity.ParentId}:{activity.Id} - Stop")
});

and then try

score 0 · Answer 2 · answered Jan 31 '22 at 16:01

There is a lot of commented code in your question which makes it a bit difficult to analyze. But it seems like you are missing starting the host after its creation, which is required for tracing provider:

Host
    .CreateDefaultBuilder(args)
    .ConfigureServices((_, services) =>
    {
        ...
        services.AddOpenTelemetryTracing(builder =>
        {
            ...
            builder.AddSource("example");
            builder.AddConsoleExporter();
        });
    })
    .Build()
    .Start();

You can do an equivalent thing fashion using Sdk.CreateTracerProviderBuilder():

Sdk
    .CreateTracerProviderBuilder()
    .AddConsoleExporter()
    .AddSource("example")
    .Build();

score 0 · Answer 3 · answered Mar 14 '23 at 16:25

It seems that listeners are automagically added, after some time. I had the exact same problem following the walkthrough at MSDN.

Failing to understand the real problem, my hack that solved the problem was overriding the worker's StartAsync method and simply adding an await Task.Delay(500).

Suddenly, the ActivitySource always creates non-null activities.

Adding Opentelemetry to .Net Core worker service

3 Answers3