3

I recently had the privilege of setting $| = 1; inside my Perl script to help it talk faster with another application across a pipe.

I'm curious as to why this is not the default setting. In other words, what do I lose out on if my buffer gets flushed straightaway?

Zaid
  • 36,680
  • 16
  • 86
  • 155
  • 2
    Faster? No, probably not, because it will be woken up more often. It will have lower latency, though. – ikegami Sep 13 '11 at 16:48
  • I often wish the same. I'm, not convinced the benefit is worth all the trouble it gives. – ikegami Sep 13 '11 at 16:50

3 Answers3

13

Writing to a file descriptor is done via system calls, and system calls are slow.

Buffering a stream and flushing it only once some amount of data has been written is a way to save some system calls.

Arnaud Le Blanc
  • 98,321
  • 23
  • 206
  • 194
  • 6
    And packets. And disk writes. And ... pretty much anything that works better with more than a few bytes of data at a time :) – Brian Roach Sep 13 '11 at 16:23
8

Benchmark it and you will understand.

Buffered depends on the device type of the output handle: ttys are line-buffered; pipes and sockets are pipe-buffered; disks are block-buffered.

This is just basic programming. It’s not a Perl thing.

tchrist
  • 78,834
  • 30
  • 123
  • 180
3

The fewer times the I/O buffer is flushed, the faster your code is in general (since it doesn't have to make a system call as often). So your code spends more time waiting for I/O by enabling auto-flush.

In a purely network I/O-driven application, that obviously makes more sense. However, in the most common use cases, line-buffered I/O (Perl's default for TTYs) allows the program to flush the buffer less often and spend more time doing CPU work. The average user wouldn't notice the difference on a terminal or in a file.

Platinum Azure
  • 45,269
  • 12
  • 110
  • 134
  • So in other words, it makes sense to auto-flush if I'm expecting a continual stream of data through the pipe as the code won't be waiting around as much. Did I get that right? – Zaid Sep 13 '11 at 16:32
  • No, just send all the headers all at once. It does not matter to linebuffer them. Everything always reads all the headers. – tchrist Sep 13 '11 at 17:57
  • How is it better to flush after each header newline instead only at the double-newline at the end? – tchrist Sep 13 '11 at 18:00
  • 1
    A large HTTP header line like `User-Agent` is only about 120 bytes. A short line may be less than 20 bytes. With a typical MTU more than 20 HTTP header lines should easily fit in a packet. If you line buffer your HTTP headers most network stacks will spit out a packet per line. Resulting in a 20x increase in TCP/IP overhead. HTTP is an perfect example of a protocol that should not be line buffered, on the other hand IRC is and equally perfect example of a protocol that should be line buffered. – Ven'Tatsu Sep 13 '11 at 19:40