2

I have a desktop application which should be notified on any table change. So, I found only two solutions which fits well for my case: SqlDependency and SQLCLR. (I would like to know if there is better in .NET stack) I have built the both structure and made them work. I only able to compare the duration of a s̲i̲n̲gl̲e̲ response from SQL Server to the client.

SqlDependency

Duration: from 100ms to 4 secs

enter image description here

SQLCLR

Duration: from 10ms to 150ms

enter image description here

I would like this structure to be able to deal with high rate notifications*, I have read a few SO and blog posts (eg: here) and also am warned from a colleague that on mass requests SqlDependency may go wrong. Here, MS offers something which I didn't get that may be another solution to my problem.

*:Not all the time but for a season; 50-200 requests per sec on 1-2 servers.

On the basis of a high rate of notifications and in parallel with performance, which of these two should I go on with, or is there another option?

Solomon Rutzky
  • 46,688
  • 9
  • 128
  • 171
ibubi
  • 2,469
  • 3
  • 29
  • 50

1 Answers1

5

Neither SqlDependency (i.e. Query Notifications) nor SQLCLR (i.e. call a Web Service via a Trigger) is going to work for that volume of traffic (50-200 req per sec). And in fact, both options are quite dangerous at those volumes.

The advice given in both linked pages (the one on SoftwareEngineering.StackExchange.com and the TechNet article) are all much better options. The advice on Best way to get push notifications to server from ms sql database (i.e. custom queue table that is polled every few seconds) is very similar to option #1 of the Planning for Notifications TechNet article (which uses Service Broker to handle the processing of the queue).

I like the queuing idea (fully custom or using Service Broker) the best and have used fully custom queues on highly transactional systems (easily the volume you are anticipating) with much success. The pros and cons between these two options (as I see them, of course) are:

  • Service Broker
    • Pro: Existing (and proven) framework (can scale and tied into Transactions)
    • Con: not always easy to configure or administer / debug, can't easily aggregate 200 individual events in 1 second into a single message (will still be 1 message per each Trigger event)
  • Fully custom queue
    • Pro: can aggregate many simultaneous trigger events into single "message" to client (i.e. polling service picks up whatever changes happened since last polling), can make use of Change Tracking / Change Data Capture as the source of "what changed" so you might not need to build a queue table.
    • Con: Is only as scalable as you are able to make it (might be as good, or better, than Service Broker, but highly dependent on your skill and experience to achieve this), needs thorough testing of edge cases to make sure the queue processing doesn't miss, or double-count, events.

You might be able to combine Service Broker with Change Tracking / Change Detection. If there is an easy-enough way to determine the last change processed (change as noted in Change Tracking / Change Data Capture table(s)), then you can set up a SQL Server Agent job to poll every few seconds, and if you find that new changes have come in, then grab all of those changes into a single message to send to Service Broker.

Some documentation to get you started:

Solomon Rutzky
  • 46,688
  • 9
  • 128
  • 171
  • Thanks for the advices, I like the certainty, but as an average sql guy, I have difficulty to understand the concepts above. Service Broker, Change Tracking, Queuing are new to me. As per your options, could you please provide a roadmap for topics which I must get familiar with to build the structure or a tutorial that shows how we can implement it? I guess after that, answer will gain value for newbies like me. – ibubi Jun 05 '17 at 14:14
  • 1
    @ibubi Please see the two links that I just added to my answer. Both take you to main pages of larger sets of documentation on those features. Good luck!! – Solomon Rutzky Jun 05 '17 at 18:52