5

I want to create 2 applications in Python which should communicate with each other. One of these application should behave like a server and the second should be the GUI of a client. They could be run on the same system(on the same machine) or remotely and on different devices.

I want to ask you, which technology should I use - an AMQP messaging (like RabbitMQ), Twisted like server (or Tornado) or ZeroMQ and connect applications to it. In the future I would like to have some kind of authentication etc.

I have read really lot of questions and articles (like this one: Why do we need to use rabbitmq), and a lot of people are telling "rabbitmq and twisted are different". I know they are. I really love to know the differences and why one of these solutions will be superior than the other in this case.

EDIT: I want to use it with following requirements:

  1. There will be more than 1 user connected at a time - I think there will be 1 - 10 users connected to the same program and they would work collaboratively
  2. The data send are "messages" telling what user did - something like remote calls (but don't focus on that, because the GUIS can be written in different languages, so the messages will be something like json informations).
  3. The system should allow for collaborative work - so it should be as interactive as possible. (data will be send all the time when user something types or performs some action).

Additional I would love to hear why one solution would be better than the other not only in this particular case.

Community
  • 1
  • 1
Wojciech Danilo
  • 11,573
  • 17
  • 66
  • 132
  • What are your requirements: 1. Number of clients (probably only 1?) 2. Amount and type of data to send 3. How frequent do you send data – schlamar Feb 05 '13 at 16:29
  • 1
    BTW, ZeroMQ does **not** implement AMQP. – schlamar Feb 05 '13 at 16:30
  • I don't think anyone can make good suggestions without knowing more about what you need either system to do, etc., since both could conceivably work just fine – matt b Feb 05 '13 at 17:29
  • 3
    There really needs to be a "Stack Architecture". I see dozens of these questions a week that are not programming questions... they are architecture questions. I feel there is a distinction. Programming questions are "tried this ...". Architecture questions are: "How do I build ...". – Adam Gent Feb 05 '13 at 19:31
  • @shlamar: You're right, I fixed the question and added additional requirements. – Wojciech Danilo Feb 06 '13 at 09:42
  • @Adam Gent: I didnt knew about Stack Architecture - I will use it, thanks! – Wojciech Danilo Feb 06 '13 at 09:43

2 Answers2

10

Twisted is used to solve the C10k networking problem by giving you asynchronous networking through the Reactor Pattern. Its also convenient because it provides a nice concurrency abstraction as threading/concurrency in Python is not as easy as say Erlang. Consequently some people use Twisted to dispatch work tasks but thats not what it is designed for.

RabbitMQ is based on the message queue pattern. Its all about reliable message passing and is not about networking. I stress the reliable part as there are many different asynchronous networking frameworks (Vert.x for example) that provide message passing (also known as pub/sub).

More often than not most people combine the two patterns together to create a "message bus" that will work with a variety of networking needs with out unnecessary network blocking and for great integration and scalability.

The reason a "message queue" goes so well with a networking "reactor loop" is that you should not block on the reactor loop so you have to dispatch blocking work to some other process (thread, lwp, separate machine process, queue, etc..). In practice the cleanest way to do this is distributed message passing.

Based on your requirements it sounds like you should use asynchronous networking if you want the results to show up instantly and scale but you could probably get away with a simple system that just polls given you only have handful of clients. So the question is how many total users (Twisted)? And how reliable do you want the updates to be delivered (RabbitMQ)? Finally do you want your architecture to be language and platform agnostic... maybe you want to use Node.js later (focus on the message queue instead of async networking... ie RabbitMQ). Personally I would take a look at Vert.x which allows you to write in Python.

Adam Gent
  • 47,843
  • 23
  • 153
  • 203
1

When someone is telling you that Twisted and RabbitMQ is different is because compare both is like compare two things with different target.

Twisted is a asynchronous framework, like Tornadao. RabbitMQ is a message queue system. You can't compare each one straight for.

You should turn your ask into a two new questions, the first one wich protocol should I use to communicate my process ? The answer can be figure out with words like amqp, Protocol Buffers ...

And the other one, which framework should I use to write my client and server program ? Here the answer can fall on Twisted, Tornado, ....

pfreixes
  • 439
  • 2
  • 6
  • 1
    Wow, uninformative comment for the win. For what it's worth, I think this answer hits the nail on the head: Twisted for endpoints, RabbitMQ between them, understand your problem domains. – ijw Jul 21 '14 at 02:58