5

What is size of the largest message real-logic's Aeron can process?

I realize Aeron shines for small messages, however I would like to have a single protocol throughout our stack and some of our messages easily reach a size of 100Mb.

The documentation is not clear on what settings affect the answer to this question. My worry is that the default buffer settings don't allow messages of this size. Or do the buffer settings have no impact on maximum application message size?

weismat
  • 7,195
  • 3
  • 43
  • 58
Erik van Oosten
  • 1,541
  • 14
  • 16
  • Someone with more reputation, please add the 'aeron' tag. – Erik van Oosten Aug 27 '15 at 07:55
  • 1
    I am not sure if something like this already exists within Aeron, but I am pretty sure you can just packetize the message of 100Mb to whatever size. Then you do not have to worry about oversizing. I am pretty sure there is documentation out there explaining this exact situation. Good luck! – M. Carlson Feb 19 '16 at 21:23

1 Answers1

6

We discussed this issue on a recent conference with Martin Thompson, main contributor of Aeron.

Not only it is not possible to exceed the size of the page, if you take a look at the Aeron presentation slides 52 to 54, these are main reasons not to push large files:

  • page faults are almost guaranteed, if message size is close to the page size, and it will make Aeron repeat the sequence with the next page, i.e. major slowdown
  • cache churn
  • VM pressure

Long story short, split your 100M message into smaller chunks, at most 1/4 of the page size minus header, if you use IPC; or the size of MTU minus header (maximum transfer unit), if you go via UDP. It will improve throughput, remove pipe clogging and fix other issues. The logic behind MTU is, occasionally UDP packets get lost and everything is re-sent by Aeron starting with lost packet. So, you want to have your chunks fit a single packet for performance.

For maximum performance, in your application-level protocol, I would create a first packet having file metadata and length, the receiving side then creates memory-mapped file on HDD of given length and fills it up with incoming chunks, where each chunk includes offset and file ID (in case you want to push multiple files simultaneously). This way you copy from Aeron buffer to mmap and completely avoid garbage collection. I expect the performance of such file transfer to be faster than most other options.

To answer the original question: With default settings, maximum message size is 16MB minus 32 bytes header. To change it, on your media driver you can try changing aeron.term.buffer.length and aeron.term.buffer.max.length system properties (see Aeron Configuration Options). Here, term buffer corresponds to a "page" in other parts of documentation and in fact, works similarly to OS memory pages in some respects. While term.buffer.max.length configures number of buffers/pages in rotation.

Alex Pakka
  • 9,466
  • 3
  • 45
  • 69