How to communicate between ZeroMQ contexts in different Goroutines?

Question

I'm using this as boilerplate, except that in the same program I also have some goroutines that are the workers and connect to the backend endpoint, tcp://127.0.0.1:5560.

What I would like to do is have it connect through a more efficient way, such as ipc://, inproc://, or even unix sockets. I've tried those, and it didn't work. Channels is a no-go with ZeroMQ right ?

So how do I connect different goroutines with ZeroMQ contexts, without tcp ? Is there a better alternative ?

update: The code:

// Simple message queuing broker
// Same as request-reply broker but using QUEUE device
//
// Author:  Brendan Mc.
// Requires: http://github.com/alecthomas/gozmq

package main

import (
    zmq "github.com/alecthomas/gozmq"
)

func startWorker() {
    context, _ := zmq.NewContext()
    defer context.Close()

    worker, _ := context.NewSocket(zmq.REP)
    //err := worker.Connect("ipc:///backend")  // Tried it, but nothing
    //err := worker.Connect("inproc:///backend")  // Tried it, but nothing
    err := worker.Connect("tcp://127.0.0.1:5560") // this works
    if err != nil {
        fmt.Println(err)
    }

    for {
        data, err := worker.Recv(0)
        fmt.Println(string(data))
        worker.Send([]byte("I got your data"), 0)
    }
}

func main() {
    context, _ := zmq.NewContext()
    defer context.Close()

    // Socket facing clients
    frontend, _ := context.NewSocket(zmq.ROUTER)
    defer frontend.Close()
    frontend.Bind("tcp://*:5559")

    // Socket facing services
    backend, _ := context.NewSocket(zmq.DEALER)
    defer backend.Close()
    //backend.Bind("ipc:///backend")  // Tried it, but nothing
    //backend.Bind("inproc:///backend")  // Tried it, but nothing
    backend.Bind("tcp://*:5560") // this works

    for i := 0; i < 4; i++ {
        go startWorker() // Start workers in a separate goroutine
    }

    // Start built-in device
    zmq.Device(zmq.QUEUE, frontend, backend)

    // We never get here…
}

I'm not really sure what you are asking? Do you want to know if you can use other protocols with 0MQ? or Are you asking if 0MQ is safe to use from multiple goroutines? — Jeremy Wall, Oct 11 '12 at 03:10
I'm asking if I can use different protocols with 0mq, do communicate between goroutines. I can communicate between goroutines through tcp, but I can't do it with inproc or ipc. Do those work under Go's goroutines as well ? — João Pinto Jerónimo, Oct 11 '12 at 11:38
Post some code, with zmq there are a zillion little things that can cause some issues... Generically with ZMQ, you should be able to use the `inproc://` connection so long as everything is within the same process. Each goroutine would need to 'share' the same context (contexts are thread-safe) and create each inproc:// connection from it. — g19fanatic, Oct 11 '12 at 13:04
Added the code, I think the problem (and the answer to this question) is that because the goroutines are not sharing the same context, they can't connect through inproc://. Is that correct @g19fanatic ? What about ipc:// or unix sockets ? Why not those ? — João Pinto Jerónimo, Oct 11 '12 at 16:01
why do you want to use 0mq to communicate between goroutines? Why not just use channels? — Jeremy Wall, Oct 12 '12 at 02:47
Because if I have a zeromq backend endpoint I could even start another program that just spawns goroutines and connects to that backend if necessary, maybe even in another machine... it gives me freedom to scale outside of that program... I believe if I wanted that I would need "net chan" which never made it to go1 — João Pinto Jerónimo, Oct 12 '12 at 02:53
also allows you to scale to programs written in other languages — g19fanatic, Oct 14 '12 at 20:55

score 5 · Accepted Answer · edited Dec 05 '13 at 09:33

In order to use the inproc:// transport, all of the sockets need to be sharing the same Context(which is thread-safe).

Also if you're using the same Context, you do not need any backend I/O threads for ZMQ

You do not mention which OS you're running under, but the ipc:// transport is only available under most *nix. Under windows you're only able to have the following transports: tcp://, inproc://, pgm://. Check out the zmq_connect documentation for more information.

How to communicate between ZeroMQ contexts in different Goroutines?

1 Answers1