6

I'm facing a problem with a slow (custom) BizTalk adapter.

Every night, an application sends more than 10'000 messages to a MSMQ within a few minutes. Unfortunately it takes hours for BizTalk to process them.

I do not have any orchestration, just route the messages to several parties. For one party we had to develop a custom adapter, but this adapter/interface is very slow. So I think BizTalk automatically throttles down the whole application and only reads as many messages from the queue that it can send over this slow adapter.

Therefore it takes hours till the MSMQ is empty.

If I stop this slow adapter and e.g. only enable a File Adapter that writes to the local file system, it takes a few seconds to process the thousands of messages from the MSMQ.

Is it possible to tweak BizTalk to process the incoming messages faster and just throttle the outgoing messages for this send port? It's unfortunate that all other parties have to wait for the messages because of one slow party.

Thank you for any advice!

Best regards Michael

MFox
  • 215
  • 4
  • 13
  • 1
    BizTalk should work this way (i.e. independent send ports) by default, unless e.g. you have some sequencing or ordering mechanism (ordered delivery, singleton orch etc). Can you check the message publishing and delivery throttling states of your hosts when the batch arrives e.g. using [these PerfMon counters](http://msdn.microsoft.com/en-us/library/aa559893.aspx)? It is possible that BizTalk is e.g. in throttling because of high spool table usage, which is affecting all messages flowing through it. – StuartLC Sep 18 '12 at 09:19

2 Answers2

6

You may be running into a Rate Based Throttling condition (see MSDN). This occurs when the Publishing Rate (rate of messages coming in) exceeds the Delivery Rate * Rate Overdrive Factor (rate of messages sent * throttling percentage).

One simple way to avoid this throttling state is to increase your Rate Overdrive Factor within the BizTalk Host Configuration settings. This is likely not best practice, as it sounds like you would need to set the Rate Overdrive Factor to an enormously high value, which could have other repercussions.

Your other alternative, depending on how you have architected the solution, is to split your Send Port / Adapter off onto its own Host Instance. As throttling is performed on a per-Host Instance basis, splitting this specific Adapter's processing off means it will no longer impact the performance of messages being delivered to other parties via standard Adapter functionality.

Brett
  • 1,127
  • 6
  • 11
1

I can think of an architectural approach that may help resolve you issue, especially considering you comment:

Is it possible to tweak BizTalk to process the incoming messages faster and just throttle the outgoing messages for this send port? It's unfortunate that all other parties have to wait for the messages because of one slow party.

For each party, create a new MSMQ 'party queue' and write directly to that queue from the incoming (large message volume) queue - a single Receive Port/Location with multiple 'party' Send Ports subscribing to incoming messages and writing to their respective 'party queues'.

You can now read from each individual 'party queue' and use an associated Send Port (with the slow adapter if required) to send messages - this should be much quicker than using a single Send Port; you are in-effect de-coupling the receiving side from the sending side.

I would also consider reviewing your Host/Host Instance strategy to ensure you have proper separation from a throtting perspective.

Nick Heppleston
  • 1,973
  • 11
  • 18