Clearing azure service bus queue in one go

Question

We are using a service bus queue in our project. We are in need of a functionality to remove all the messages from the queue when the administrator chooses to clear the queue. I searched on the net but could not find any function which does this inside the QueueClient class.

Do I have to pop all the messages one by one and then marking them complete to clear the queue or is there a better way?

QueueClient queueClient = _messagingFactory.CreateQueueClient(
                              queueName, ReceiveMode.PeekLock);

BrokeredMessage brokeredMessage = queueClient.Receive();

while (brokeredMessage != null )
{
    brokeredMessage.Complete();
    brokeredMessage = queueClient.Receive();
}

Answer 1

You can simply do this from azure portal.

If you are not worried about queue messages stored in past 14 days (default) and wanted to clear queue immediately then change message time to live value to 5 or 10 seconds and wait for 5-10 seconds.

Important step: After a few seconds, try to receive from queue to force it to update.

All queue messages will be cleared. you can again reset original value. default is 14 days.

Answer 2

Using the Receive() method within the while loop like you are will cause your code to run indefinitely once the queue is empty, as the Receive() method will be waiting for another message to appear on the queue.

If you want this to run automatically, try using the Peek() method.

For example:

while (queueClient.Peek() != null)
{
    var brokeredMessage = queueClient.Receive();
    brokeredMessage.Complete();
}

You can make this simpler again with the ReceiveMode.ReceiveAndDelete as was mentioned by hocho.

Answer 3

Using :

• Both approach (from @ScottBrady and @participant)
• And the MessageReceiver abstraction

you can write a method that empty a service bus queue or a topic/subscription:

MessageReceiver messageReceiver = ...
while (messageReceiver.Peek() != null)
{
    // Batch the receive operation
    var brokeredMessages = messageReceiver.ReceiveBatch(300);

    // Complete the messages
    var completeTasks = brokeredMessages.Select(m => Task.Run(() => m.Complete())).ToArray();

    // Wait for the tasks to complete. 
    Task.WaitAll(completeTasks);
}

Answer 4

The first I thought I had when I read the question was why not delete and recreate the queue? That could be an approach. But if don't want to do that, you can receive and delete each message until there's no message left.

You can use the latest Azure Service Bus .NET library Azure.Messaging.ServiceBus for both approaches.

1. Delete and recreate queue using AdministrationClient

using Azure.Messaging.ServiceBus.Administration;

ServiceBusAdministrationClient adminClient = new ServiceBusAdministrationClient("<connectionstring>");
await adminClient.DeleteQueueAsync("myqueue");          
await adminClient.CreateQueueAsync("myqueue");          

Note: ServiceBusAdministrationClient is for CRUD operations on an already existing Service Bus namespace. If you also need the ability to create a namespace, use the Microsoft.Azure.Management.ServiceBus library instead.

2. Receive and delete until there's no message

using Azure.Messaging.ServiceBus;

await using var client = new ServiceBusClient("<connectionstring>");
                
ServiceBusReceiver receiver = client.CreateReceiver("myqueue", 
    new ServiceBusReceiverOptions { ReceiveMode = ServiceBusReceiveMode.ReceiveAndDelete });

while ((await receiver.PeekMessageAsync()) != null)
{
    // receive in batches of 100 messages.
    await receiver.ReceiveMessagesAsync(100);
}

ReceiveMode = ServiceBusReceiveMode.ReceiveAndDelete is self-explanatory. The default receive mode is PeekLock which doesn't actually remove messages from the queue - it tells Service Bus that the receiving client wants to settle the received messages explicitly. The lock part means competing receivers can't access the message for the duration of the lock.

Answer 5

I'm having good results using a combination of ReceiveAndDelete, PrefetchCount, ReceiveBatchAsync, and a simple truth loop rather than using Peek. Example with MessagingFactory below:

var receiverFactory = MessagingFactory.CreateFromConnectionString("ConnString");
var receiver = receiverFactory.CreateMessageReceiver("QName", ReceiveMode.ReceiveAndDelete);
receiver.PrefetchCount = 300;

bool loop = true;
while (loop)
{
    var messages = await receiver.ReceiveBatchAsync(300, TimeSpan.FromSeconds(1));
    loop = messages.Any();
}

Only requires the WindowsAzure.ServiceBus Nuget package.

Answer 6

For Azure-ServiceBus-Queues there is a ReceiveBatch-method which allows you to receive a batch of n-messages at the time. Combined with ReceiveMode.ReceiveAndDelete you can clear the queue more efficiently.

Caveat The number n of messages might be returned but it's not guaranteed. Also there is a limit for the message-batch size of 256K.

Answer 7

The fastest way to clean a Azure ServiceBus Queue is to set a very short DefaultMessageTimeToLive, wait a few second, try receive from queue from force it to update then restore the initial DefaultMessageTimeToLive.

You can do it from the portal or from code :

var namespaceManager = NamespaceManager.CreateFromConnectionString(connectionString);
var queueDescription = _namespaceManager.GetQueue(queueName);
var queueClient = QueueClient.CreateFromConnectionString(connectionString, queueName, ReceiveMode.ReceiveAndDelete);

var dl = queueDescription.EnableDeadLetteringOnMessageExpiration;
var ttl = queueDescription.DefaultMessageTimeToLive;

queueDescription.EnableDeadLetteringOnMessageExpiration = false;
queueDescription.DefaultMessageTimeToLive = TimeSpan.FromSeconds(1);

Thread.Sleep(5000);
var dumy = queueClient.ReceiveBatch(200, TimeSpan.FromSeconds(1)).ToArray();

queueDescription.EnableDeadLetteringOnMessageExpiration = dl;
queueDescription.DefaultMessageTimeToLive = ttl;

Answer 8

There is a simple Clear() method to clear the entire queue if you are using the WindowsAzure.Storage library from nuget. I use the Microsoft.Windows.Azure.Queue class from that library to manage the queue. Otherwise, you can access via their API per their documentation. I don't know how long the method has been in the Azure library and it probably didn't exist when the question was originally asked, but the REST API stems back from at least 2014 per this Azure feedback post

The full .NET code to clear the queue with the Azure library is:

string connectionString = "YourAzureConnectionStringHere";
string queueName = "YourWebJobQueueName";
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connectionString);

// Create the queue client, then get a reference to queue
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
queue = queueClient.GetQueueReference(GetQueueName(queueName));

// Clear the entire queue
queue.Clear();

Answer 9

1. Use Queue Service REST API (not tested).

The Clear Messages operation deletes all messages from the specified queue

DELETE https://myaccount.queue.core.windows.net/myqueue/messages

Only the account owner may call this operation.

PS. The URL seems to be wrong. Try this instead: https://myaccount.servicebus.windows.net/myqueue/messages

Source: https://learn.microsoft.com/en-us/rest/api/storageservices/clear-messages

Or...

2. Create a logic app to delete messages one by one in a loop (tested).