3

This may be a stupid question but there's a OneToOneRingBuffer and a ManyToOneRingBuffer available in aeron (agrona). I have a single producer and many consumers I'm wondering how to achieve the equivalent effect of OneToManyRingBuffer?

hawk
  • 1,827
  • 2
  • 14
  • 28

1 Answers1

3

BroadcastTransmitter and BroadcastReceiver in Agrona is one approach that would give you one-to-many semantics.

Worth noting that slow consumers may see messages dropped. If this is not desired then you could also look at Aeron IPC which would exert back-pressure on the publisher in this scenario.

Example

Top-level, here's how to use it:

// Create broadcast buffer
int capacity = 1 << 10; // Must be power of two
int bufferSize = capacity + BroadcastBufferDescriptor.TRAILER_LENGTH;
UnsafeBuffer broadcastBuffer = new UnsafeBuffer(new byte[bufferSize]);

// Create transmitter
BroadcastTransmitter transmitter = new BroadcastTransmitter(broadcastBuffer);

// Create receiver (can create many of these)
BroadcastReceiver broadcastReceiver = new BroadcastReceiver(broadcastBuffer);
CopyBroadcastReceiver copyBroadcastReceiver = new CopyBroadcastReceiver(broadcastReceiver);

// Send message
int msgTypeId = 1;
MutableDirectBuffer msgBuffer = new ExpandableArrayBuffer();
int msgLength = msgBuffer.putStringWithoutLengthAscii(0, "Hello World!");
transmitter.transmit(msgTypeId, msgBuffer, 0, msgLength);

// Receive message
copyBroadcastReceiver.receive(
  (msgType, buffer, offset, length) -> System.out.println(buffer.getStringWithoutLengthAscii(offset, length)));
James
  • 388
  • 3
  • 7
  • I don't have the option of applying back-pressure on the producer. I'd like slow consumers to transparently switch to reading from from file (ArchiveMedia?) when they're behind and go back to reading from the buffer when they've caught up. – hawk Jun 15 '20 at 08:09
  • That's possible with Aeron. Are you doing this over the network or on the same host? – James Jun 15 '20 at 11:42
  • Same host possibly different NUMA node (and I imagine on another host I could easily setup a copy of the buffer with a "producer" blindly picking up the messages and dumping onto the buffer) – hawk Jun 15 '20 at 17:42