Why and how RabbitMQ .NET Core client causes the application shutdown

Question

I create the client as follows (the exchangeAndQueue2 are durable and exist):

var factory = new ConnectionFactory() {
    HostName = hostname,
    Port = port,
    UserName = userName,
    Password = password
};
factory.AutomaticRecoveryEnabled = true;
factory.NetworkRecoveryInterval = TimeSpan.FromSeconds(10);
connection = factory.CreateConnection();
channel = connection.CreateModel();
channel.ExchangeDeclare(exchange1, ExchangeType.Fanout, durable: true);
consumer = new EventingBasicConsumer(channel);
consumer.Registered += (s, e) => { Trace.TraceInformation("Consumer Registered"); };
consumer.ConsumerCancelled += (s, e) => { Trace.TraceInformation("Consumer Cancelled"); };
consumer.Received += NewMessage;
channel.BasicConsume(exchangeAndQueue2, autoAck: false, consumer: consumer);

Most of the time it works fine, but SOMETIMES my whole application shuts down silently because of the RabbitMQ. I subscribed to the FirstChanceException and ProcessExit events

AppDomain.CurrentDomain.FirstChanceException += (sender, eventArgs) => { Trace.TraceError(eventArgs.Exception.ToString()); };
AppDomain.CurrentDomain.ProcessExit += (sender, eventArgs) => {
    Trace.TraceWarning("Application is shutting down...");
    mq.CloseConnection();
}

And here's what I managed to catch:

Server Error: 0 : System.Net.Sockets.SocketException (10060): A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
Server Error: 0 : System.IO.IOException: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
Server Error: 0 : System.Net.Sockets.SocketException (10060): A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
   at RabbitMQ.Client.Impl.InboundFrame.ReadFrom(NetworkBinaryReader reader)
Server Information: 0 : RabbitMQ Consumer Cancelled
Server Warning: 0 : Application is shutting down...

What could it be and how to debug such things? How is it even possible that a library shuts down the whole app? The whole code of the NewMessage handler is wrapped in try/catch (Exception ex)...

Server version 3.1.5, Client version 5.1.0

My Main method ends with the channel.BasicConsume call, looks like I don't understand how to organize a bulletproof reception.

I guess the application exits when the receiver thread quits... How to properly respawn it?

It's a long operation inside try/catch with Acknowledgement or Rejection in the end. Anyway, I think I figured out what's wrong and going to post the answer — Himura, Dec 27 '18 at 11:54
I saw your answer. Yes, exiting the `Main` will cause an exit- indeed, that's the very definition of exiting `Main`. — theMayer, Dec 29 '18 at 15:22
The main issue is that the exit will not happen immediately, the application will wait for other threads to exit (I guess) before closing completely. And the RabbitMQ message listener thread can survive for quite a long periods. So long that I did not even notice that the Main method returns right after the app starts. — Himura, Dec 30 '18 at 16:06

Himura · Answer 1 · 2018-12-27T13:37:34.353

According to the RabbitMQ documentation, I should not exit the Main method. They use ReadLine, but I prefer catching Ctrl+C like in this answer:

static void Main(string[] args) {
    //...
    channel.BasicConsume(exchangeAndQueue2, autoAck: false, consumer: consumer);
    Trace.TraceInformation("Application started. Press Ctrl+C to shut down.");
    var exitEvent = new AutoResetEvent(false);
    Console.CancelKeyPress += (s, e) => { e.Cancel = true; exitEvent.Set(); };
    exitEvent.WaitOne();
    Trace.TraceInformation("Ctrl+C pressed.");
    mq.CloseConnection();
}

Why and how RabbitMQ .NET Core client causes the application shutdown

1 Answers1