5

I'm running 2 multi-threaded programs.

Each thread from the first program acts as a producer and writes messages to a queue, whereas each thread from the second program acts as a consumer and reads messages from the same queue.

In both projects I created a connection factory like the following:

ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
factory.setAutomaticRecoveryEnabled(true);
factory.setRequestedChannelMax(0);
factory.setUsername("user");
factory.setPassword("password");

However I'm not sure about the recommended approach for the next step.

  1. Should I create a new connection at the start of each thread like:

    Connection connection = factory.newConnection();

    And then for each request create a new channel like:

    Channel channel = connection.createChannel();

  2. Or should I create only a single connection, make the threads share the same connection, and then create a new channel for every request.

I know that the connection is a socket thread-safe connection and it should be created carefully. I'm just asking about whether there is a recommended approach to use in my program, because usually a documentation would contain a recommended way to handle connections and sockets, but I couldn't find such an answer in RabbitMQ's documentation.

Said A. Sryheni
  • 697
  • 1
  • 9
  • 29
  • Please read [Why is asking a question on “best practice” a bad thing?](https://meta.stackexchange.com/questions/142353/why-is-asking-a-question-on-best-practice-a-bad-thing/243450) before attempting to ask more questions that are opinion based that invite argumentative discussion because they do not have a single agreed upon answer. –  Dec 19 '17 at 14:44
  • 1
    Usually I would read the documentation which would state what is the recommended approach. When I couldn't find such an answer I wrote a question to see if maybe there is a standard practice to do one of the 2 choices I presented. Anyways I will consider your comment the next time I write a question. – Said A. Sryheni Dec 19 '17 at 14:53
  • 1
    Re : Close votes - Rabbit's connection handling is somewhat different from many other TCP/IP based protocols, in that Rabbit explicitly multiplexes logical message transfer across a single TCP/IP connection, in order to reduce the overall number of connections to the server cluster, and thus has a documented preference for a single, [long lived](https://github.com/squaremo/amqp.node/issues/46) Connection per client process, meaning the answer is unlikely to be primarily opinion based. – StuartLC Dec 20 '17 at 07:02

2 Answers2

5

Use your Option 2. Rabbit MQ is fine with sharing a single Connection per process, so unless you are connecting to multiple brokers or have an extreme load profile, a single, long lived Connection shared across all threads in your process will suffice. Connections are relatively expensive (80% of the way down) to obtain, so short lived Connections will hamper producer performance.

Connections are thread safe - internally, the RabbitMQ client multiplexes multiple channels across a single Connection, allowing for concurrent use of the connection.

i.e. You can use the Connection as a "Channel Factory". However, Channels aren't thread safe, so you will typically create a short lived channel, produce a message, and close the channel on the producer (Channels are cheap to obtain).

At a suggestion, on the producer, instead of sending messages directly to a queue, instead, publish messages to an Exchange (i.e. Skip to the 'publish-subscribe' tutorial). This way you can make use of additional routing topologies, beyond the classical point-to-point mechanisms used by older middleware such as MSMQ / MQSeries.

StuartLC
  • 104,537
  • 17
  • 209
  • 285
  • Thanks for your reply. First, I'm publishing my messages to an Exchange, I just didn't mention it because I didn't want to make my question more complex. Second, In my situation threads are created once the program starts, and are not supposed to terminate unless the while program stops. Also, speed is a very important factor in my case. Does this change anything? I mean would it affect the overall efficiency if I created a connection per thread? – Said A. Sryheni Dec 19 '17 at 14:35
  • 1
    Great, re exchange (but you did mention `writes messages to a queue`). Although you could create a Connection per thread if each thread, it is common place just to create a single Connection (or abstraction) and then share it, e.g. by registering it with an IoC container. Long lived producer threads isn't a problem - every time each needs to send a message, it will just create a channel off the Connection. Channels are cheap, so I don't think there's much benefit in 'long lived' channels. – StuartLC Dec 19 '17 at 14:40
  • I'm not planning to create a 'long lived' channels. In fact in both options I will create a 'short lived' channels. The difference is whether should I obtain this channel from a shared connection? or from a connection that is only dedicated to the corresponding thread? Will I gain any performance improvement from creating a connection per thread? – Said A. Sryheni Dec 19 '17 at 14:44
  • 1
    I can't find definitive literature on the performance benefits of multiple connections, but the general rule of thumb appears to be a [single connection](https://derickbailey.com/2014/03/26/2-lessons-learned-and-3-resources-for-for-learning-rabbitmq-on-nodejs/) for most scenarios. You also need to weigh up the downside that on the server side, that additional connections will eat more server resources. If you have extreme load conditions, you might find that Rabbit is the wrong tool for the job, e.g. Apache Kafka might be better suited. – StuartLC Dec 19 '17 at 15:00
  • 1
    Thank you very much, I got everything I need now. – Said A. Sryheni Dec 19 '17 at 15:10
-1

As described in documentation:

  • For Connection:

Current implementations are thread-safe for code at the client API level, and in fact thread-safe internally except for code within RPC calls. https://www.rabbitmq.com/releases/rabbitmq-java-client/current-javadoc/com/rabbitmq/client/Connection.html

  • For Channel:

As a rule of thumb, sharing Channel instances between threads is something to be avoided. Applications should prefer using a Channel per thread instead of sharing the same Channel across multiple threads.

https://www.rabbitmq.com/api-guide.html section Channels and Concurrency Considerations (Thread Safety)

So, Connection is thread safe, Channel is not

user1516873
  • 5,060
  • 2
  • 37
  • 56
  • My question wasn't actually based on the idea whether a connection is thread safe or not, I already know the answer to this question. My concerns are about increasing the efficiency of my program. – Said A. Sryheni Dec 19 '17 at 14:40
  • Write test and check it. It is impossibe to answer without knowing your system parameters. And even if I know it, still writing perfomance test is preferrable way. Results should be close for small thead count, i think. – user1516873 Dec 19 '17 at 14:54