11

I'm using MailboxProcessor classes in order to keep separate agents that do their own thing. Normally agents can communicate with one another in the same process, but I want agents to talk to one another when they are on separate processes or even different machines. What kind of mechanism is best for implementing communication between them? Is there some standard solution?

Please note that I'm using Ubuntu instances to run the agents.

Timothy Baldridge
  • 10,455
  • 1
  • 44
  • 80
Dmitri Nesteruk
  • 23,067
  • 22
  • 97
  • 166
  • 7
    Who voted this as 'off-topic'? If you don't understand the question please don't interfere with it. – Mauricio Scheffer Jan 31 '11 at 16:02
  • 1
    http://stackoverflow.com/questions/501656/f-mailbox-processor-on-distributed-systems may provide some insight. – Greg Campbell Jan 31 '11 at 16:51
  • maybe look at windows azure queues. Its what they use to let different machines (they call them worker roles) talk to each other. What you can do is have an agent on each machine that is in charge of communicating with the others to either request work, notify work is done etc.. – jlezard Feb 08 '11 at 16:28
  • @jlezard interesting, but I'm not using Azure. I've already figured out that I need to set up my own type of queuing infrastructure – Dmitri Nesteruk Feb 15 '11 at 14:22

1 Answers1

3

I think you're going to have write your own routines to serialize messages, pass them accross the process boundaries and then dispatch them on the other side. This will also require a implementation of a ID system where each mailbox has an ID and processes can send messages to IDs instead of just Mailbox.Send. This is not easy, as local boxes will be able to access local memory, but remote mailboxes will not.

I would look at something like RPyC (http://rpyc.wikidot.com/) as it provides a protocol somewhat like you are looking for.

Basically the answer is 'no' there isn't really a good way to do this.

Timothy Baldridge
  • 10,455
  • 1
  • 44
  • 80
  • do you know how this would go in other languages which have agents: Erlang, Scala and co? Thanks! – jlezard Feb 08 '11 at 16:21
  • Well, in Erlang, you don't have shared memory. Each "thread" is really a separate process. So when you say `pid ! "foo"` pid is simply a id of a process that `"foo"` gets sent to. Since there is no shared memory, and processes have pids already, networking it is as simple as creating a dictionary of pid to IP mappings and passing messages based on that information. The problem with this method is that all communication has to be done via message passing, and currently also involves a memory copy of the message. Performance-wise this isn't always best. I don't have any experience with Scala. – Timothy Baldridge Feb 08 '11 at 16:28
  • And I should claify, Erlang processes are not OS processes, they are VM processes. A Erlang process has a memory overhead of a few hundred bytes. A OS process has an overhead in the multi-KB range. – Timothy Baldridge Feb 08 '11 at 16:29