12

What is a good solution for communication via message broker that supports both (C)Python and Java/JMS applications? My particular requirements are:

  • open source solution
  • Available on Linux-based systems
  • No rendezvous between sender and receiver required (i.e. uses a message broker)
  • Multiple producers and consumers supported for a single event queue (only one consumer receives each message)
  • Unit of work support with two-phase commit (XA support nice to have)
  • Support for persistent messages (i.e. that survive a restart of the broker)
  • Supports JMS for Java clients
  • No component is "fringe", meaning at risk of falling out of maintenance due to lack of community support/interest
  • If there is a Python client that manages to "speak JMS" that would be awesome, but an answer including a task to write my own Python JMS layer is acceptable

I have had a surprisingly hard time finding a solution for this. Apache's ActiveMQ has no Python support out of the box. ZeroMQ requires a rendezvous. RabbitMQ does not appear to support JMS. The best candidate I have found is a combination of ActiveMQ and the pyactivemq library. But the first and last release of pyactivemq was in 2008, so it would appear that that fails my "no fringe" requirement.

The ideal answer will be the names of one or more well-supported and well-documented open source packages, that you have personally used to communicate between a Java/JMS and Python application, and that do not require a lot of integration work to get started. An answer that includes an "easy" (up to a few days of work) implementation of additional glue code to meet all the requirements above, would be acceptable. A commercial solution, in the absence of a good open source candidate, would be acceptable also.

Also, Jython is out. (If only I could...) The same Python applications will need to use modules only available in CPython.

wberry
  • 18,519
  • 8
  • 53
  • 85
  • Amazon SQS or SNS supports Java & Python... http://code.google.com/p/awspylib/ – sdolgy Jul 01 '11 at 19:21
  • 1
    Not my area of expertise but [ZeroC's Ice](http://www.zeroc.com/) may have what your looking for. – geaw35 Jul 01 '11 at 20:03
  • @sdolgy I guess I should have stated as a requirement that I don't want to trust the contents of messages to a third party or send them over the internet. Nonetheless, an interesting platform. – wberry Jul 01 '11 at 21:24
  • @mrt It seems that Ice could be used to implement a general message broker (and a lot of other things). Not what I would call an "easy" DIY step to have the solution I need, since I would need to implement unit-of-work, message persistence and multiple producers/consumers. Thanks though, I did not know about this before and it looks very mature. – wberry Jul 01 '11 at 21:29
  • Was going to ask "Have you looked into Jython?" until I noticed your final paragraph. Oh well. – JAB Jul 11 '11 at 16:04

4 Answers4

5

JMS is a specification not implementation . RabbitMQ is a really option .

I have also happily used HornetQ http://www.jboss.org/hornetq from Jboss as with every thing it is more aligned with every thing Java EE but RabbitMQ would be choice espcially if you are using Spring as well

Arjan Tijms
  • 37,782
  • 12
  • 108
  • 140
Shahzeb
  • 4,745
  • 4
  • 27
  • 40
  • Clarification on my JMS compatibility requirement. Because JMS allows properties and headers to be transmitted, as well as either text or bytes messages, then these mandatory features in effect place requirements on the underlying transport. A Python peer, and the API it uses, must therefore somehow support headers and byte/character messages in order to communicate with a Java JMS peer. I do not expect a Python peer to support serialized Java object messages, however. – wberry Jul 14 '11 at 13:49
4

I have had a surprisingly hard time finding a solution for this. Apache's ActiveMQ has no Python support out of the box.

ActiveMQ brokers fully support using the Stomp protocol out of the box. Stomp is a text based protocol for messaging that has clients for many platforms and languages.

ActiveMQ's documentation should contain information on how to set up a connector for stomp. In its simplest form, enabling a connector would look something like:

<transportConnectors>
   <transportConnector name="stomp" uri="stomp://localhost:61613"/>
</transportConnectors>

Once enabled on the broker side, you can then use any python library that supports stomp. You can then use Stomp on the python side and JMS on the java side for communication with the broker and sending/receiving from specific destinations.

whaley
  • 16,075
  • 10
  • 57
  • 68
  • I haven't completed my evaluation of all the answers to date, but so far this one is the favorite, as ActiveMQ is looked on more favorably by my organization. I will evaluate all these answers and issue/award new bounties to any others that prove feasible. Thanks all. – wberry Jul 18 '11 at 14:22
1

You might want to take a look at OpenAMQ and another look at RabbitMQ.

The underlying messaging technology used by RabbitMQ and OpenAMQ is AMQP. You should be able to easily find Python and Java clients that work against both of these brokers (and ostensibly any other spec-compliant broker).

If JMS is a must-have, then you might be able to find a JMS client out there implemented on top of AMQP (OpenAMQ provided such a client at one time, but I am unsure of its current status).

Brandon E Taylor
  • 24,881
  • 6
  • 47
  • 71
  • The promised JMS support in RabbitMQ isn't materializing. Its literature says JMS is supported, but I can't find any occurrences of the string 'javax.jms' in any of the source, even for the 'rabbitmq-java-client' project. There are a number of references to a project called openamq-jms, which would have allowed JMS support for both brokers, but all the links to it are broken as of now. How were you able to get JMS working with these brokers? – wberry Jul 11 '11 at 15:50
  • Regardless, +1 for pointing me to OpenAMQ which I had not found earlier even with considerable searching. – wberry Jul 11 '11 at 15:52
1

We had been using GlassFish Message Queue (formerly Sun Java MQ) - it is inherited from OpenMQ

It satisfies most of your requirements, if not all. We had been using fail over-clustered brokers in Red Hat Linux (RHEL) - it is reliable for heavy usage. Though some 'quirks' lurk here and there.

ring bearer
  • 20,383
  • 7
  • 59
  • 72
  • This one's starting to look pretty good. Open source, JMS and STOMP. +1 and I'll evaluate before awarding the bounty. – wberry Jul 11 '11 at 16:18