32

To read data from a socket in python, you call socket.recv, which has this signature:

socket.recv(bufsize[, flags])

The python docs for socket.recv vaguely state:

Note: For best match with hardware and network realities, the value of bufsize should be a relatively small power of 2, for example, 4096.

Question: What does "best match with hardware and network realities" mean? What is the actual impact of setting bufsize to a non-power-of-two?

I've seen many other recommendations to make this read a power of 2. I'm also well aware of reasons when it is often useful to have array lengths as powers of two (bitshift/masking operations on the length, optimal FFT array size, etc), but these are application dependent. I just am not seeing the general reason for it with socket.recv. Certainly not to the point of the specific recommendation in the python documentation. I also don't see any power-of-two optimizations in the underlying python code to make it a python-specific recommendation

For example... if you have a protocol where the incoming packet length is exactly known, it is obviously preferrable to only read "at most" what is needed for the packet you are dealing with, otherwise you could potentially eat into the next packet and that would be irritating. If the packet I'm currently processing only has 42 bytes pending, I'm only going to set bufsize to 42.

What am I missing? When I have to choose an arbitrary buffer/array size I usually (always?) make the length a power of two, just in case. This is just a habit developed over many years. Are the python docs also just a victim of habit?

This isn't exclusive to python, but since I'm specifically referencing the python docs I'll tag it as such.

UPDATE: I just checked the size of the buffer at the kernel level on my system (or at least I think I did... I did cat /proc/sys/net/core/rmem_default) and it was 124928. Not a power of two. rmem_max was 131071, also clearly not a power of two.

In looking into this more I really cannot see any benefit in the power of two recommendation(s) yet. I'm about ready to call it as a bogus recommendation...

I also added tcp and C tags since they are also relevant.

Community
  • 1
  • 1
Russ
  • 10,835
  • 12
  • 42
  • 57
  • My best guess is that some OS-level network stacks may prefer to maintain power-of-2-sized buffers for memory management reasons, but this does seem like a pointless historical artifact. Maybe the main thrust of the note is that the buffer size should be on the order of 4KB rather than 4MB or 4GB, and the power-of-2 is just superstition. – Russell Borogove Jun 15 '11 at 23:15
  • 1
    My guess is that the only possible real consequence of a non-power-of-two size is that some lower level of code might allocate a buffer whose size is the next power of two anyway, and the latter portion if the buffer wouldn't get used. Not really a problem unless you are insanely short on RAM... – Jeremy Friesner Jun 16 '11 at 02:25
  • Most likely the only possibly-important numbers are: cache line size (usually 64 bytes, but always a power of 2), and page size (usually 4096 bytes, but always a power of 2). As long as you're a *multiple* of whichever of those is more relevant, you're probably ideal. – o11c Oct 01 '17 at 18:52
  • Note also that due to cache association weirdness, it's often better to *not* use exactly a power of 2. – o11c Oct 01 '17 at 18:53
  • Have a look at the answers on [this SO question](https://stackoverflow.com/questions/1708835/receving-socket-python). It deals with `recv()` too and the answers may be helpfull for a more complete understanding. It won't answer the question but will provide a bit more insight. – extraneon Jun 16 '11 at 19:56

2 Answers2

9

I'm pretty sure the 'power of 2' advice is based on an error in editing, and should not be taken as a requirement.

That specific piece of advice was added to the Python 2.5 documentation (and backported to Python 2.4.3 docs), in response to Python issue #756104. The reporter was using an unreasonably large buffer size for socket.recv(), which prompted the update.

It was Tim Peters that introduced the 'power of 2' concept:

I expect you're the only person in history to try passing such a large value to recv() -- even if it worked, you'd almost certainly run out of memory trying to allocate buffer space for 1.9GB. sockets are a low-level facility, and it's common to pass a relatively small power of 2 (for best match with hardware and network realities).

(Bold emphasis mine). I've worked with Tim and he has a huge amount of experience with network programming and hardware, so generally speaking I'd take him on his word when making a remark like that. He was particularly 'fond' of the Windows 95 stack, he called it his canary in a coalmine for its ability to fail under stress. But note that he says it is common, not that it is required to use a power of 2.

It was that wording that then led to the documentation update:

This is a documentation bug; something the user should be "warned" about.

This caught me once, and two different persons asked about this in #python, so maybe we should put something like the following in the recv() docs.

"""
For best match with hardware and network realities, the
value of "buffer" should be a relatively small power of 2,
for example, 4096.
"""

If you think the wording is right, just assign the bug to me, I'll take care of it.

No one challenged the 'power of 2' assertion here, but the editor moved from it is common to should be in the space of a few replies.

To me, those proposing the documentation update were more concerned with making sure you use a small buffer, and not whether or not it is a power of 2. That's not to say it is not good advice however; any low-level buffer that interacts with the kernel benefits with alignment with the kernel data structures.

But although there may well be an esoteric stack where buffers with a size that is a power of 2 matters even more, I doubt Tim Peters ever meant for his experience (that it is common practice) to be cast in such iron-clad terms. Just ignore it if a different buffer size makes more sense for your specific use cases.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • Great answer and a fun piece of history. Thanks!! – Russ Oct 03 '17 at 13:04
  • 5
    Ya, "power of 2" was a red herring here. A specific socket implementation may or may not favor particular buffer sizes over others, but the real point of that "bug" report resolution was to discourage the poster from trying to use multi-gigabyte buffers. BTW, the value of Win95 was to discover bugs in Python (& Zope). Its threads & sockets were fine, but had radically different pragmatics (like timing) than Unix-y systems of the time. Running stress tests on Win95 uncovered a world of legit bugs! – Tim Peters Oct 04 '17 at 02:32
3

In regards to: "if you have a protocol where the incoming packet length is exactly known, it is obviously preferrable to only read "at most" what is needed for the packet you are dealing with, otherwise you could potentially eat into the next packet and that would be irritating."

This may be preferable for the application developer, but is probably inefficient for the underlying network stack. First, it ties up socket buffer space that can be used for additional network I/Os. Second, each recv() you make means dipping into a system call/kernel space and there is a performance penalty for the transition. It is always preferable to get as much data as you can out of kernel space and into user space with as few system calls as possible and do your message parsing there. This adds more complexity to the application code and message handling but is probably the most efficient.

That said, given the speed of today's processors and amount of available memory, this may not be an issue for most applications, but this was a common recommendation for network applications back in the "old days".

I am not sure about the power of 2 recommendation from a user-space application. I have seen these types requirements for drivers due to alignment and page size issues, etc. but its not clear what effect this has from user space unless it somehow aids in copying data out of kernel buffers into user buffers. Maybe somebody with more OS development knowledge could comment.

ribram
  • 2,392
  • 1
  • 18
  • 20
  • Good response! But... python's `bufsize` parameter doesn't have anything to do with the underlying kernel level socket buffer. `bufsize` is just size of the buffer (a python string) that is allocated to send to the kernel recv call to stuff the read bytes into. ie: `bufsize` should not be "inefficient for the underlying network stack". Although it does highlight that if socket.recv is returning prematurely (slowly/patchily arriving incoming data??), the user-space buffer allocation may be excessive. I the kernel buffer has all bytes, it limits the kernel recv calls to one, which is nice. – Russ Jun 17 '11 at 16:08
  • Also of interest is that I just checked (I think) the size of the kernel recv buffer on my system and it is an odd value... 124928. I'm going to update the question with that since the non-power-of-two there also seems relevant. – Russ Jun 17 '11 at 16:10
  • 2
    Yes, recv() bufsize has nothing to do with kernel buffer size. The inefficiency is in (a) leaving data lingering in kernel buffers means less available network buffer space for the kernel at any point in time. Ideally you want to free kernel buffers and perform message parsing in your user buffers and (b) extraneous system calls are inefficient. Say 2 messages of 42 bytes are sent to you and waiting. It is better to do a single recv(), get 84 bytes and parse the messages in your buffers than 2 recv() of 42. For two messages it doesn't matter but becomes a question of scale. – ribram Jun 17 '11 at 16:21
  • I get it now... thanks for clarifying. I cringe at the idea of over-reading and having a second layer of buffering to parse at the application level, but that is mostly rooted in the sometimes-irritating string immutability causing extra space usage. Maybe I'll fiddle with it and use a python buffer (now called memoryview I guess... like the new name) to try and avoid that. – Russ Jun 17 '11 at 16:34