How do I stop NServiceBus retrying a message that HASN'T thrown an exception?

Question

I'm experiencing behavior related to retries that I can't find any reference to in documentation or searches I've tried. Essentially if my handler takes longer than 60 seconds to process a message (note it does NOT throw an exception), NServiceBus fires another handler to process the same message. This means the work done by the handler runs (at least) twice (often 5 times as I have MaxRetries configured to 5).

I only want the message to be retried if it really fails (i.e. an exception is thrown), not just because its taking longer than a minute.

Is this behavior as designed? Can it be turned off? Is the "60 seconds" configurable?

In the logs it just looks like a new message being handled:

2014-12-09 14:50:38,406 [13] DEBUG NServiceBus.Pipeline.BehaviorChain`1 - ChildContainerBehavior
2014-12-09 14:50:38,422 [13] DEBUG NServiceBus.Pipeline.BehaviorChain`1 - MessageHandlingLoggingBehavior
2014-12-09 14:50:38,430 [13] DEBUG NServiceBus.Unicast.Behaviors.MessageHandlingLoggingBehavior - Received message with ID 031e6070-4397-4e55-8670-a3fc00f49d7c from sender Foo
2014-12-09 14:50:38,440 [13] DEBUG NServiceBus.Pipeline.BehaviorChain`1 - ImpersonateSenderBehavior
...
2014-12-09 14:50:40,313 [13] DEBUG NServiceBus.Pipeline.BehaviorChain`1 - InvokeHandlersBehavior
2014-12-09 14:50:40,319 [13] INFO  MyHandler - Running
...
...
2014-12-09 14:51:38,642 [15] DEBUG NServiceBus.Pipeline.BehaviorChain`1 - ChildContainerBehavior
2014-12-09 14:51:38,667 [15] DEBUG NServiceBus.Pipeline.BehaviorChain`1 - MessageHandlingLoggingBehavior
2014-12-09 14:51:38,678 [15] DEBUG NServiceBus.Unicast.Behaviors.MessageHandlingLoggingBehavior - Received message with ID 031e6070-4397-4e55-8670-a3fc00f49d7c from sender Foo
2014-12-09 14:51:38,686 [15] DEBUG NServiceBus.Pipeline.BehaviorChain`1 - ImpersonateSenderBehavior
...
2014-12-09 14:51:38,831 [15] DEBUG NServiceBus.Pipeline.BehaviorChain`1 - InvokeHandlersBehavior
2014-12-09 14:51:38,837 [15] INFO  MyHandler - Running

Take a look at the endpoint logs, there is the possibility that what is happening is that the transaction timeouts (the default timeout is 1 minute) does an exception is basically thrown under the hood and the message retried. — Mauro Servienti, Dec 09 '14 at 11:33
All I see in the log (which I've added above) is the pipeline behavior chain running all the behaviors, as if a new message has been received. Nothing about a transaction timeout. — zeroid, Dec 09 '14 at 17:14

score 1 · Accepted Answer · answered Dec 16 '14 at 13:25

Mauro was on the right track, but the logs didn't show it very well. We're using SQL server for the message queues. Transactions by default will timeout after 60 seconds, but the timeout doesn't immediately cause any error. Only when the message handling completes will NServiceBus attempt to update/move the completed message, but it can't because the transaction is no longer valid (an exception is then thrown).

The reason a second handler kicks off after only 60 seconds is because concurrent processing is enabled, and once the transaction times out for the first handler, the message effectively becomes available for another thread to pick up (its locked until this point).

So the solution is to add something like this in config:

  <system.transactions>
    <defaultSettings timeout="00:10:00" />
  </system.transactions>

(Presumably if you are using MSMQ as the message transport, a different timeout may apply...)

Woah! You got your own answer in one hour before mine! (: But do have a look at my remark about the machine config so you don't accidentally fall into the (almost) same trap again! — Reyhn, Dec 16 '14 at 15:05

score 1 · Answer 2 · edited May 23 '17 at 11:59

I had the exact same symptoms, and the problem was long-running transactions via DTC (distributed transactions).

As Mauro suggests, the DTC has a default timeout of 60 seconds. This can be changed either on the system:

Start the "Component Services", then expand "Component Service -> Computers -> My Computer" and right-click and select Properties. On the Options-tab you can set the desired default timeout.

Alternatively, you can change it using the app.config:

<configuration>
  <system.transactions>
    <defaultSettings timeout="00:10:00"/>
  </system.transactions>
</configuration>

However, there is a caveat here. The default system timeout is still set to 10 minutes! If you know you will have transactions lasting for more than 10 minutes, you need to add the following to your machine.config:

<configuration>
  <system.transactions>
    <machineSettings maxTimeout="01:00:00" />
  </system.transactions>
</configuration>

The machine.config is located at

%windir%\Microsoft.NET\Framework64\[version]\config\machine.config

References:

DTC troubles with long running transactions in NServiceBus

Override the System.Transactions default timeout of 10 minutes in the code

Where is my machine.config?

If anyone knows how to get NServiceBus to **not** hide this error, please leave a comment. The fact that there is a problem at all is not very obvious at first. — Reyhn, Dec 16 '14 at 15:06

score 0 · Answer 3 · answered Jul 22 '15 at 22:23

For those who simply doesn't want NServiceBus to mess around with transaction stuff (a.k.a. get off my transactions way, dude) one can simply make NSB stop managing handlers in a transaction scope:

//e.g. When using the BusConfiguration object:
 config.Transactions()
     .DoNotWrapHandlersExecutionInATransactionScope();

How do I stop NServiceBus retrying a message that HASN'T thrown an exception?

3 Answers3