22

I want to find a queue structure (a data container) whose elements must be first-in-first-out. It is important for me that the structure must be thread-safe. I'm going to use this data container as something like a task or connection pool.

I know a buffered channel is thread-safe, but I wonder if it works as FIFO, especially in a concurrent situation.

And if it is possible to use buffered channel as a thread-safe queue, do I need to worry about its efficiency?

Mostafa
  • 26,886
  • 10
  • 57
  • 52
hardPass
  • 19,033
  • 19
  • 40
  • 42
  • 1
    Channels are the answer. There's little chance of finding or writing something better than channels for such a task. – Mostafa May 03 '12 at 04:33
  • 1
    Channels are especially suited to tasks queues, resources sharing, connection pools, and the like. Be careful not to reproduce the patterns of a thread based language as one of the big progress of the Go language is the goroutine+channel mechanism. Be sure to understand it (you probably can submit an architecture or strategy to SO). – Denys Séguret May 03 '12 at 05:39
  • 1
    This is the first thing I thought of when I learned about channels in Go, thanks for asking this! – thanos Aug 12 '15 at 07:47
  • But the are a question, I think if use golang channels will be at most times, less secure, standard and complete than use a message broker like RabbitMQ or ActiveMQ with the same effort at the end. So I do not think that is a compensative aproach – deFreitas Mar 27 '17 at 23:17

3 Answers3

19

In Go, a buffered channel is just that: a thread-safe FIFO queue so what you are trying to do is perfectly valid. You shouldn't have performance issues at all with this approach.

  • 1
    `The Go Programming Language` asks to avoid use Buffered Channels as queues and use slices instead. – piepi May 01 '22 at 02:58
  • 1
    @piepi could you link to this by chance? I've never heard of this. It's hard to avoid using Buffered Channels as queues, because that's effectively what they are... I'm wondering if there's more nuance here. – tsturzl Oct 07 '22 at 15:16
  • @tsturzl "Novices are sometimes tempted to use buffered channels within a single goroutine as a queue, lured by their pleasingly simple syntax, but this is a mistake . Channels are deeply connected to goroutine scheduling, and without another goroutine receiving from the channel,a sender—and perhaps the whole program—risks becoming blocked forever. If all you need is a simple queue, make one using a slice." P233 – stonecutter Jul 27 '23 at 03:04
  • The case of the book refer to is where only one goroutine in the whole program. – stonecutter Jul 27 '23 at 08:46
  • @stonecutter Thinking of how queues generally work in other languages, such as C++, the same is still true. If you're implementing a queue to feed a pool of threads, if the sender stops sending the receivers are usually block indefinitely on a cond_var. You can notify_all the cond_var to get the threads unstuck to do some kind of cleanup logic, which is synonymous to having a context or a control channel in golang with a select statement. So really depends on what you're using the queue for, if it's a work stealing queue, use a channel. – tsturzl Aug 01 '23 at 19:15
  • I suppose many of us are probably assuming the need for this queue is likely work stealing which could be wrong, but that said it's really situational weather or not you'd be concerned with the way a buffered channel works. Empty channel blocks receivers, full channel blocks senders. If that behavior is undesirable, probably don't use a channel. – tsturzl Aug 01 '23 at 19:16
5

I'm pretty sure that Channels are FIFO. They are also cheap so they would be memory efficient. Beyond that without knowing the details of how you are going to use them We can't really give much more advice.

Jeremy Wall
  • 23,907
  • 5
  • 55
  • 73
  • Thanks. So, the Channels are both FIFO and Thread-Safe? Would you please give me a link or something about the implementation details of Channel's mechanism? – hardPass May 03 '12 at 04:50
  • The definitive documentation would be: http://code.google.com/p/go/source/browse/src/pkg/runtime/chan.c See the implementation of chansend and chanrecv for the details. The specification does not specifically say they are a fifo queue but it's derivable I think from the details of how to use them. – Jeremy Wall May 03 '12 at 14:06
  • But the are a question, I think if use golang channels will be at most times, less secure, standard and complete than use a message broker like RabbitMQ or ActiveMQ with the same effort at the end. So I do not think that is a compensative aproach – deFreitas Mar 27 '17 at 23:17
2

In general, I would say buffered channels do not make a good concurrency-safe queue. Creating them allocates memory for the entire buffer. If your queue size varies from very small to very large during execution, you have to allocate for the worst case scenario and may be wasting a lot of memory.

Anthony V
  • 2,179
  • 1
  • 11
  • 6