Azure Service Bus Topic Subscriber receiving order

Question

I am using azure service bus topic as a message broker for my solution. As per my understanding for each subscription the Azure Message Bus keeps a virtual queue, so at receiving end order of messages shouldn't be disturbed.

But in reality it is bit different, In my scenario

The input is roughly after every two seconds, (The timestamp is correct I have verified it)
If I disconnect the receiver for sometime, the messages start to queue up on Azure against the subscription.
Then If I connect receiver again, the receiving code receives messages quite quickly, but the order is not maintained?
However If I keep the client connected, the messages are received in order (1 message after two seconds)

Receiving code

SubscriptionClient Client = SubscriptionClient.CreateFromConnectionString(connectionString, topicname, subscription_name);

                // Configure the callback options.
                OnMessageOptions options = new OnMessageOptions();
                options.AutoComplete = false;
                options.AutoRenewTimeout = TimeSpan.FromMinutes(1);

                // Callback to handle received messages.
                Client.OnMessage((message) =>
                {
                    try
                    {
                        // Process message from queue.
                        string payload = message.GetBody<string>();
                        var myData = JsonConvert.DeserializeObject<MyData>(payload);
                        if(myData != null)
                        {

                            //Timestamp is not in order, when I connect after few minutes
                            Debug.WriteLine("SBC ==> " + myData.Timestamp);

                        }
                        // Remove message from queue.
                        message.Complete();
                    }
                    catch (Exception)
                    {
                        // Indicates a problem, unlock message in queue.
                        message.Abandon();
                    }
                }, options);

Output

SBC ==> 5/23/2017 1:06:43 PM
SBC ==> 5/23/2017 1:06:45 PM
SBC ==> 5/23/2017 1:07:23 PM
SBC ==> 5/23/2017 1:07:19 PM
SBC ==> 5/23/2017 1:07:27 PM
SBC ==> 5/23/2017 1:07:07 PM
SBC ==> 5/23/2017 1:06:49 PM
SBC ==> 5/23/2017 1:07:47 PM
SBC ==> 5/23/2017 1:06:47 PM
SBC ==> 5/23/2017 1:08:03 PM
SBC ==> 5/23/2017 1:06:55 PM
SBC ==> 5/23/2017 1:06:51 PM
SBC ==> 5/23/2017 1:07:03 PM
SBC ==> 5/23/2017 1:07:51 PM
SBC ==> 5/23/2017 1:06:57 PM
SBC ==> 5/23/2017 1:07:05 PM
SBC ==> 5/23/2017 1:07:39 PM
SBC ==> 5/23/2017 1:07:43 PM
SBC ==> 5/23/2017 1:06:59 PM
SBC ==> 5/23/2017 1:07:09 PM
SBC ==> 5/23/2017 1:06:53 PM
SBC ==> 5/23/2017 1:07:33 PM
SBC ==> 5/23/2017 1:07:25 PM
SBC ==> 5/23/2017 1:07:57 PM
SBC ==> 5/23/2017 1:08:13 PM

Can any one please explain why it is so? It is bit confusing for me?

NPhillips · Answer 1 · 2017-05-23T15:07:17.357

3

While azure service bus offers itself as FIFO(First in First out) this only really works during a uninterrupted session. As you experienced with:

However If I keep the client connected, the messages are received in order

To get around this you can use a different mode. Take a look at ReceiveAndDelete and PeekLock modes in the below link.

Service Bus Docs

Here are some relevant stack overflow posts that may help you.

How to accomplish FIFO with Azure service bus topics

How to gurantee azure queue FIFO

EDIT

This link contains some details on the FIFO

Below is a quote from that Doc that specifies you need to use messaging sessions in order to get FIFO.

The guaranteed FIFO pattern in Service Bus queues requires the use of messaging sessions. In the event that the application crashes while processing a message received in the Peek & Lock mode, the next time a queue receiver accepts a messaging session, it will start with the failed message after its time-to-live (TTL) period expires.

Documents seem to be fairly lacking on implementing message sessions but from my understanding it is from MessageSession class, and it's the AcceptMessageSessionmethod

edited May 23 '17 at 15:07

answered May 23 '17 at 14:24

NPhillips

505
4
19

Thanks for your comments, Could you please refer to some documentation on MSDN where this `The problem you are having is that when you disconnect the receiver the messages that time out begin to retry after a set amount of time. This causes them to lose the order they were sent in.` is described? It would be really helpful. – Muhammad Ummar May 23 '17 at 14:46
Will add to the answer. – NPhillips May 23 '17 at 14:55
Have removed the part you quoted from the question as am unable to provide a solid source for it. I had this issue a while ago and that was my understanding of the issue but of course my word counts for little without docs to back it up. You may find this blog post useful though I suspect it is a little out of date: http://www.ben-morris.com/dont-assume-message-ordering-in-azure-service-bus/ – NPhillips May 23 '17 at 15:11

score 3 · Answer 2 · answered May 23 '17 at 15:22

Just like @NPhillips said, you need to use Message Sessions feature of the ASB to achieve FIFO behavior. This means several things, which are important to note:

A receiver will only process a single session a time
Concurrent processing is not possible, you'll be down to a single message per whatever time it take to process.
Sender needs to assign session ID to each message.

The best sample and explanation would be the one posted by the ASB team on GitHub here.

Azure Service Bus Topic Subscriber receiving order

2 Answers2