2

Setup

I have a model of a distributed system in which there is a producer(P), a consumer(C) and 1, 2, 3, ... n workers(Wn). All of these components are communicating via the Microsoft Azure Service Bus(B). Inside the bus, there is a topic(T) and a queue(Q).

The (P) is at varying rates pushing messages into the (T). The (Wn)'s [number of them is a consequence of the rate of (P)'s messages] are fetching these messages from there, altering them according to some pre-defined function and after this forwards the messages to the (Q), from which the (C) picks them up and handles them according to plan.

Purpose

The purpose of this model is to investigate the scalability of a system like this, with specific regards taken to the Azure Service Bus. The applications themselves are written in C# and they are all executed from within the same system.

Questions

I have two concerns in regard to the functionality of the Azure Service Bus:

  1. Is there a way to tell either the (B) to be more loose in terms of balancing, or perhaps make the (W)'s more 'eager' to participate?

There seems to be a pre-destined order of message distribution, making the load balance uneven (among the (W)'s).

Say for instance I have 3 (W)'s - or (W3): if (P) now were to send 1.000 messages to the (T), I would expect a somewhat even distribution, going towards 1/3 of all messages for each of the (W). This is however not the case; it seems as if the rest of the (W)'s just sits there waiting for the busy (W) to handle message after message after message. Suddenly, perhaps after 15 to 20 messages, another (W) will receive a message, but still the balance is very uneven.

Consequently, I now have (W)'s just sitting around doing nothing (for varying periods of time).

  1. Is there a way, either in (B)'s settings or (W)'s code, to specifically set the time of the PeekLock()?

I have experimented with Thread.Sleep(timeToSleep) in the (W) OnMessage()-function. This seems to fit my needs, if it wasn't for the concern aired in the first question.

My experimentation: whenever a message arrives at a (W), the work begins and just before message.Complete() is sent to the (B), I pull off a Thread.Sleep(2000) or something along that line. Ideally, another (W) should pick up where this first (W) fell asleep, but they don't. The first (W) wakes up and grabs another message and so the cycle continues, sometimes 15-20 times until another (W) finally grabs a message.

Images

If you excuse my poor effort at explaining through drawing, this is current scenario (figure 1) and the ideal, wanted, scenario (figure 2):

Figure 1: current scenario

Figure 2: optimal/wanted scenario

I hope for some clarification on this matter. Thank you in advance!

Community
  • 1
  • 1
H. Larsson
  • 31
  • 3

1 Answers1

0

The distribution of messages across consumers is handled in the about order that requests for messages are being made to Service Bus. There is no assurance for excatly even distribution at the message level, and the distribution will be affected by feature usage, including prefetch. In any actual workload situation, you'll find that distribution is fair, because busy workers will not ask for more messages.

Clemens Vasters
  • 2,666
  • 16
  • 28