Getting Message Stats in Azure Service Bus

Question

I am writing a utility to monitor our Azure Service Bus Topics and Subscriptions.

I can get the Topic details, such as name, queued message count and dead-letter message count, but I would like to get the number of messages that have been processed.

Here is the code I am using:

var sub = namespaceManager.GetSubscription(topicPath, subscriptionName);

var name = sub.Name;
var pending= sub.MessageCountDetails.ActiveMessageCount;
var deadletter = sub.MessageCountDetails.DeadLetterMessageCount

It seems that GetSubscription does not include any properties to get the number of messages processed.

Has anyone tried to do this before?

Answer 1

To get messages stats from Azure Servicebus entities, I use Visual Studio App Insights. This is a tool to monitor apps. Basically, your app sends events to App Insights and from the Azure Portal, you can create dashboards that give you real-time information on your app.

To monitor Azure Servicebus entities, I send custom events from my app:

• You can have a look at the pricing, there is a free plan that allow you to send up to 5 millions of custom events per month. If you need to send more than 5 millions events, you can create an App Insights per Servicebus entity or aggregate count before sending events to App Insights.

• You have access to your the raw data for 7 days and aggregate data for 90 days.

• If you use Power BI, you can configure a continuous export of your data (don't think it is available in the free plan).

• Other cool thing, you can send exceptions and create alert from App Insigths that send you an email any time exceptions are received to App Insigths.

If you process servicebus messages from a webjob/worker role/console app/windows service, this article could be a god starting point :

• Monitoring usage and performance in Windows Desktop apps

So after creating an App Insights from the Azure portal, you will get an InstrumentationKey.

You can install ApplicationInsights from nuget.

To send event to App Insights, you need to instantiate a TelemetryClient. Microsoft recommands to only have once instance of the telemetry client per app and to flush the TelemetryClient when the app stops or restarts:

var telemetryClient = new TelemetryClient()
    { InstrumentationKey = "MyInstrumentationKey" };

So this is a very basic example but you'll get the point:

// Get the message
BrokeredMessage message = ...

try
{
    // Process you message
    ...

    // Delete the message from the queue when it is ok.
    message.Complete();

    // Create and send an event to app insights
    var eventTelemetry = new EventTelemetry { Name = "MyQueueName" };
    eventTelemetry.Metrics["MessageCount"] = 1;
    telemetryClient.TrackEvent(eventTelemetry);
}
catch (Exception ex)
{
    // Send back the message to the queue ??? depends if you'd like to re-process it
    message.Abandon();

    // Send the exception to app insights
    telemetryClient.TrackException(ex);
}

Using this code, you will have a new event in App Insights with name MyQueueName. You can create a dashboard and filter on this event and display the MessageCount metric. I use a metric because in more complex scenario, you can just send one event every x minutes and set the MessageCount to the number of messages processed in this interval.

Here I am using App insights but I am pretty sure you can do the same with other tools like :

• New Relic or
• Raygun

Hope this will help you !

Answer 2

It is possible to fetch the Total Count of Messages in a Topic, Incoming Messages, Outgoing Messages with the help of the latest Azure Monitor Metrics. In your case, count of Outgoing messages will be the number of processed messages.

Answer 3

Number of processed messages since creation of the entity? Or from a certain date? And what about messages that have been processed multiple times (delivery count > 1) because the client failed to complete on time or some other reasons? This count is not straight forward and that's why is not available out of the box.