30

From PyPubSub:

Pypubsub provides a simple way for your Python application to decouple its components: parts of your application can publish messages (with or without data) and other parts can subscribe/receive them. This allows message "senders" and message "listeners" to be unaware of each other:

  • one doesn't need to import the other
  • a sender doesn't need to know
    • "who" gets the messages,
    • what the listeners will do with the data,
    • or even if any listener will get the message data.
  • similarly, listeners don't need to worry about where messages come from.

This is a great tool for implementing a Model-View-Controller architecture or any similar architecture that promotes decoupling of its components.

There seem to be quite a few Python modules for publishing/subscribing floating around the web, from PyPubSub, to PyDispatcher to simple "home-cooked" classes.

Are there specific advantages and disadvantages when comparing different different modules? Which sets of modules have been benchmarked and compared?

Thanks in advance

Oliver
  • 27,510
  • 9
  • 72
  • 103
Eli Bendersky
  • 263,248
  • 89
  • 350
  • 412

7 Answers7

16

PyDispatcher is used heavily in Django and it's working perfectly for me (and for whole Django community, I guess).

As I remember, there are some performance issues:

  • Arguments checking made by PyDispatcher is slow.
  • Unused connections have unnecessary overhead.

AFAIK it's very unlikely you will run into this issues in a small-to-medium sized application. So these issues may not concern you. If you think you need every pound of performance (premature optimization is the root of all evil!), you can look at modifications done to PyDispatcher in Django.

Hope this helps.

zuber
  • 3,449
  • 2
  • 24
  • 19
4

The best dispatch package for python seems to be the dispatch module inside django (called signals in the documentation). It is independent of the rest of django, and is short, documented, tested and very well written.

Edit: I forked this project into an independent signal project for Python.

Olivier Verdier
  • 46,998
  • 29
  • 98
  • 90
  • "It is independent of the rest of django... " Apparently not: https://github.com/django/django/blob/master/django/dispatch/dispatcher.py#L71 – Nick Zalutskiy Jun 14 '11 at 21:45
  • 1
    It *is* independent of the rest of django. Remove the line you highlighted and you obtain this independent package: https://github.com/olivierverdier/dispatch – Olivier Verdier Jun 15 '11 at 20:09
  • In my search I also came across this: https://github.com/theojulienne/PySignals which seems to offer the same thing. And this: https://github.com/11craft/louie which seems to be a "more complicated" descendent of PyDispatch (that I couldn't find any documentation for). I intended to research this, but if you have a minute, what is the difference between these projects? – Nick Zalutskiy Jun 15 '11 at 21:45
  • It seems that the PySignals fork is more recent. It includes patches to make the module "more thread safe." https://github.com/django/django/commit/f793de865612626272a90ef805e2adde9035667d Also, PySignals includes the `reciever` decorator thats mentioned in the Django docs: https://docs.djangoproject.com/en/1.3/topics/signals/#connecting-receiver-functions – Nick Zalutskiy Jun 17 '11 at 17:38
3

Here is a newer one: https://github.com/shaunduncan/smokesignal. "smokesignal is a simple python library for sending and receiving signals. It draws some inspiration from the django signal framework but is meant as a general purpose variant." Example:

from time import sleep
import smokesignal

@smokesignal.on('debug')
def verbose(val):
    print "#", val


def main():
    for i in range(100):
        if i and i%10==0:
            smokesignal.emit('debug', i)
        sleep(.1)

main()
Jabba
  • 19,598
  • 6
  • 52
  • 45
2

I recently looked carefully at py-amqplib to act as an AMQP client to a RabbitMQ broker. The latter tool is written in Erlang.

If you're looking to decouple your app. then why couple it to the language itself? Consider using message queues which are language neutral and then you've really got room to grow!

That being said, AMQP takes effort to understand and may be more than you are willing to take on if your app. is working just fine as is. YMMV.

Mike
  • 3,663
  • 3
  • 20
  • 12
  • I'm a big fan of message queues, but sometimes it's less about decoupling an application from itself so much as you just need another way for different parts of the app to talk to itself, and a simple, in-process solution is a better fit than the heavy handed option of a real messaging middleware service. One advantage of choosing the in-process version is that it's still fairly easy to add the remote messaging later by adding a message subscriber that then serializes the message and passes it to the broker, or connects to the broker and publishes the unserialized message back in process. – SingleNegationElimination Jul 21 '11 at 22:52
2

Some libraries I have found that haven't yet been mentioned:

M.Babcock
  • 18,753
  • 6
  • 54
  • 84
cmcginty
  • 113,384
  • 42
  • 163
  • 163
1

There is also the libraries by PJ Eby, RuleDispatch and the PEAK project, specially Trellis. I don't know what their status actually but the mailing list is quite active.

Last version of Trellis on PyPi

Trellis doc

I have also used the components from the Kamaelia project of the BBC. Axon is an interesting approach, but more component than publisher-consumer inspired. Well, its website is somewhat not up-to-date at all... There was a project or 2 in the Google SoC 2008 and work is being done.

Don't know if it help :)

Edit : I just found Py-notify which is an "unorthodox" implementation of the Observer pattern. It has most of the functionalities that I need for my own tools.

edomaur
  • 1,397
  • 4
  • 23
  • 38
0

The fact alone that PyPubSub seems to be a somewhat chaotically managed project (the Wiki on SF is dead, the website (another Wiki) which is linked on SF is currently broken) would be enough reason for me not to use it. PyDispatcher has an intact website, but the only documentation they seem to provide is the one for the API generated from the docstrings. No traffic on the mailing list either... a bad sign!

As Mike also mentioned, it's perfectly possible to choose a solution that is independent of Python. Now don't get me wrong, I love Python, but still, in this field it can make sense use a framework that is decoupled from the programming language.

I'm not experienced with messaging, but I'm planning to have a look into a few solutions. So far these two (free, open source) projects seem to be the most promising for me (coincidentally, both are Apache projects):

Both seem to be reasonably matured projects, at least a far as documentation and community. I can't comment on the software's quality though, as I said, I didn't use any of the software.

Qpid ships with client libraries for Python, but you could also use py-amqplib. For ActiveMQ there's pyactivemq, which you can use to connect either via STOMP (Streaming Text Orientated Messaging Protocol) or via Openwire.

paprika
  • 2,424
  • 26
  • 46
  • Note that the PyPubSub you are referring to in your answer is clearly not the one in the OP. The OP was referring to pubsub.sf.net which I eventually moved to github.com/schollii/pypubsub a few years after the OP. Also pypubsub.sf.net used to be the dead project you refer to, unrelated to mine but I eventually was given the right to take over the domain name for my PyPubSub. Both pypubsub.sf.net and pubsub.sf.net have been pointing to the same PyPubSub (mine) for a few years, but at time of OP the pypubsub.sf.net was wrong one. See the gihub link above for updates. – Oliver Mar 24 '19 at 22:06