Why would you use a message based system?

Question

What are the motivations for using a message based system?

I'm seeing a lot about service buses such as NServiceBus and Mass Transit and I'm wondering what the benefits of the underlying methodology are.

Answer 1

There are multiple advantages to using message based systems.

1. Messages form a well defined technology neutral interface between applications.
2. Enables loose coupling of applications.
3. Lots of options for performance, tuning and scaling:
   • Deploy requester and service process on different hardware
   • Multiple requesters sharing single server
   • Multiple requesters sharing multiple servers
4. The various messaging middlewares implement the common messaging patterns independently from you application.
   • Request/Reply
   • Fire and Forget offline updates
   • Publish/Subscribe
5. Many of the middleware products handle message transformation (e.g. SWIFT to SWIFTXML).
6. Many of the middleware products can decompose a single large request into several smaller requests.
7. They nearly all support multiple platforms.

Incidentally the two market leaders in this area are IBM with their Websphere MQ and related products, and, TIBCO with their Enterprise Service Bus.

Answer 2

A message-based architecture de-couples producers and consumers of messages, both in time and space. This has a lot of benefits:

• producers and consumers can run on different machines
• producers and consumers can run at different times.
• producers and consumers can run on different hardware/software platforms (they only need to understand the same message protocol)
• it's easy to coordinate multiple producers / consumers (e.g. for compute-intensive jobs that need multiple machines, as Dave Markle has described)
• higher stability when services are temporarily unavailble (e.g. when doing order processing, using a messaging system can help avoid "dropping" orders)

You lose most of these benefits when you do RPC-style communication (i.e. when you block while waiting for service responses)

Answer 3

One use case is when you have a pool of resources that can work on a given item, and a list of work which needs to be distributed in a scalable fashion.

I once had a project where I had to do mainframe integration with a number of 3270 screen scrapers (all slow). I could have at most 10 of these processes open on a box at a time. I had thousands of accounts to screen-scrape and update, so I put the work in a queue, and my machines (I had about 3 of em) just picked up work items off the message queue (MSMQ) and did their screen scraping, and that was it. I could easily spin up a new machine, or deactivate old ones without interrupting the flow of work, so that was nice.

Answer 4

the benefits are really down to decoupling the parts of your application. Once you have the bus set up, and applications are added, you can easily extend your app by adding new pieces that you can guarantee will not affect the other parts. Its a very good way of continually adding to a system over time.

eg. we have a system like this, each command is implemented as a part to the overall GUI, if we want to add a new feature, or modify an existing one, we just write a new part and add it to the system. When it gets called, it has no dependencies on the rest. It means we can extend our app very easily. Also, we have a message passing between nodes on our network - when something gets changed in 1 computer, a message is sent to all others so they can update themselves. We have a hundred different messages for different events, so we can extend the system by dropping a new service in that reacts to the appropriate messages.

Message passing architectures typically have the same features as web services, you have discrete services you can call, you can add new ones easily.

Don't think that message passing architectures require fancy (and expensive!) middleware products though, Windows runs on a message passing architecture - every WM_* message passed to a window is.. well, a message, and I think that shows the best example of the architecture - no part of a system needs to know about any other part, you can extend it infinitely as you can handle as many controls as you like on any dialog, etc etc.

Message passing is a fabulous architecture, though it can be slower than tightly coupling your application together, that's not much of a reason not to use it nowadays especially if you're already using scripting or .net apps.

Answer 5

I helped develop one for a system that used C# and Remoting, the client (a service or GUI) could send a message complete with some custom data and the recipient(s) would receive it, either when they next connected or instantly). They could then process the message (by taking ownerships of it in the case of the load-balancing services). It was also used to update GUIs when long running processes had finished.

The messaging system itself had configurable pipelines that processed each message, here are some examples of a few of the pipelines we developed:

• Storage (created/ saved the message to the database)
• Routing (worked out where the message needs to be sent)
• Security (worked out if the client was allowed to receive it)
• Delete (works out if the message needs to be deleted from the system, which means clients need to be notified and the message removed from it's cache and flagged in the DB).
• TakeOwnership (deals with making sure only one client can modify the state of a message at a time as only messages that are owned can be changed)

So in answer to your question, messaging systems are a brilliant means of sending information about when you don't know or care the who client is.

Answer 6

Message oriented system are generally good for certain classes of integration problems. Other alternatives are having a shared data store (file or DB based perhaps) for applications to communicate with or applications integrating via RPC.

The advantages of messaging over these integration patterns are that you aren't coupling both applications to the same data store schema and you aren't tying applications to a point to point RPC integration scenario (which gets more complex the more applications are involved).

There's also the benefits of asynchronous communication (like email vs. online chat) and message routing and transformation possibilities.

Answer 7

A message-based system can lay the groundwork for the entire digital side of your business. These days, having digital solutions that are seamlessly integrated is paramount for collecting, storing, and analyzing data, and it's crucial that your data help you make the most informed decisions possible when it comes to your business processes. Essentially, a message-based service is a form of API (application programming interface) that can send requests in the form of messages from one application to another. Using a message-based system means that all of your digital solutions can be connected easily, and they'll communicate reliably with each other.

Requests can be sent to any endpoint within the system, and once verified, they can be sent through multiple pathways simultaneously. This greatly speeds up processes compared to a traditional request/respond model which would only be able to use one pathway at a time. With a service like TIBCO messaging, you can communicate across a wide variety of APIs, support mobile and IoT devices, scale your messaging service to your needs, and simplify application development and adoption. Naturally, the platform provides a secure messaging environment as well, so you can rest assured that your data is only shared between your own applications for your specified purposes.

Answer 8

There is one main issue, and that is of reliability. In distributed systems, sync calls can fail of downstream system is getting overloaded. Downstream systems all would then need to be tuned to load coming from upstream, which is impractical. Messaging systems can take care of spikes. Even better is to use load balancers in addition to messaging system. While downstream application is autoscaling up (which can take a few minutes), the messaging system can take care of reliably storing spikes. Also, messaging system and do pub - sub reliably. Imagine if there are thousands of trades coming and to be dispatched to hundreds of traders.. easily done with pub - sub. So operative word is Reliability.

Answer 9

I would suggest to have a look on this blog post . It shows the end to end workflow and explains how we can integrate the messaging platform programmatically.

https://eduwebmonster.com/implementing-message-based-communication-in-modern-application-architecture/