63

We have a daemon that contains a lot of print messages. Since we are working on an embedded device with a weak CPU and other constraint hardware, we want to minimize any kinds of costs (IO, CPU, etc..) of printf messages in our final version. (Users don't have a console)

My teammate and I have a disagreement. He thinks we can just redirect everything to /dev/null. It won't cost any IO so affections will be minimal. But I think it will still cost CPU and we better define a macro for printf so we can rewrite "printf" (maybe just return).

So I need some opinions about who is right. Will Linux be smart enough to optimize printf? I really doubt it.

iBug
  • 35,554
  • 7
  • 89
  • 134
Michael
  • 1,313
  • 11
  • 25
  • 18
    Beware side effects: `printf("%d", x=a+b);` If you redirect to */dev/null* side effects will happen; if you rewrite as a *do nothing* macro, side effects will be lost – pmg Jan 15 '19 at 09:42
  • 5
    Providing a `myprintf(...) { return; }` is probably what you want. You can then have a macro for printf forwarding to that method, preserving side effects yet not formatting any string or calling write – msrd0 Jan 15 '19 at 10:10
  • put the printf stuff in a preprocessor block and turn it off for production – Vorsprung Jan 15 '19 at 13:52
  • 15
    @pmg: Side-effects in a `printf` statement are evil. In code review, I'd definitely raise an issue over that. – MSalters Jan 15 '19 at 14:37
  • 8
    I would take a step back. In a Linux daemon there are far better options then printf... Consider for example using syslog instead, if you set the log level at startup (ideally from an environment variable) and you can direct the logs to file or to another machine over the network trivially, and the log level allows you to turn off the logging of things you don't care about at relatively low cost in execution time. Even better if you do something like trapping a few signals you can arrange to change the log level at runtime without stopping, much less having to recompile the daemon. – Dan Mills Jan 15 '19 at 19:33
  • 1
    Related: https://stackoverflow.com/questions/44613418/how-does-dev-null-eat-up-output-streams /dev/null just ignores the buffer. `printf` still has to do the work of writing to it. – cs95 Jan 15 '19 at 20:18
  • 4
    Seems like you need a proper logging framework. At the minimum, one that supports lazy evaluation of the message. – Alexander Jan 15 '19 at 23:48
  • 1
    The answers are maybe a bit naive about the correct choice, though the information is correct. 1) the penalty for IO *can* be gigantic on weird embedded systems (and with pathological logging), so you may recover much of the cost despite the printf cost; 2) the cost may be so small that it's better just to redirect to /dev/null for simplicity and to reduce your workload as developers. – Dannie Jan 16 '19 at 11:48
  • @DanMills You left out how using `printf()` for logging ties the deamon to the output file selected at startup - for the life of the daemon process. `printf()` output is also buffered, which means abnormal termination of a process leads to lost log entries - right when you *need* to see exactly what's going on. That's two *more* reasons `printf()` is a very, very, very bad logging mechanism. :-) – Andrew Henle Jan 17 '19 at 11:53
  • 1
    @AndrewHenle True, and the fact that it is holding a file descriptor to a disk file open means that logrotate or whatever cannot drop the reference count to zero and free the disk space. You can unlink the filename, but until the daemon terminates the space used on disk will just keep on growing. Printf sucks as a logging mechanism, and odds are syslog is there in the system, so may as well use it. – Dan Mills Jan 17 '19 at 12:59
  • Your teammate is lacking in the most basic understanding of how computers, programs, operating systems, and libraries work. – Jim Balter Jan 18 '19 at 02:59
  • 1
    'printf() output is also buffered' -- only if one is foolish enough not to flush it. – Jim Balter Jan 18 '19 at 04:32

6 Answers6

74

Pretty much.

When you redirect the stdout of the program to /dev/null, any call to printf(3) will still evaluate all the arguments, and the string formatting process will still take place before calling write(2), which writes the full formatted string to the standard output of the process. It's at the kernel level that the data isn't written to disk, but discarded by the handler associated with the special device /dev/null.

So at the very best, you won't bypass or evade the overhead of evaluating the arguments and passing them to printf, the string formatting job behind printf, and at least one system call to actually write the data, just by redirecting stdout to /dev/null. Well, that's a true difference on Linux. The implementation just returns the number of bytes you wanted to write (specified by the 3rd argument of your call to write(2)) and ignores everything else (see this answer). Depending on the amount of data you're writing, and the speed of the target device (disk or terminal), the difference in performance may vary a lot. On embedded systems, generally speaking, cutting off the disk write by redirecting to /dev/null can save quite some system resources for a non-trivial amount of written data.

Although in theory, the program could detect /dev/null and perform some optimizations within the restrictions of standards they comply to (ISO C and POSIX), based on general understanding of common implementations, they practically don't (i.e. I am unaware of any Unix or Linux system doing so).

The POSIX standard mandates writing to the standard output for any call to printf(3), so it's not standard-conforming to suppress the call to write(2) depending on the associated file descriptors. For more details about POSIX requirements, you can read Damon's answer. Oh, and a quick note: All Linux distros are practically POSIX-compliant, despite not being certified to be so.

Be aware that if you replace printf completely, some side effects may go wrong, for example printf("%d%n", a++, &b). If you really need to suppress the output depending on the program execution environment, consider setting a global flag and wrap up printf to check the flag before printing — it isn't going to slow down the program to an extent where the performance loss is visible, as a single condition check is much faster than calling printf and doing all the string formatting.

Prakash Pazhanisamy
  • 997
  • 1
  • 15
  • 25
iBug
  • 35,554
  • 7
  • 89
  • 134
  • 31
    Answers like this should state they are based on general understanding of common implementations and not upon specific documentation. In theory, there is no reason a C implementation might not inspect `stdout`, learn it is /dev/null, and suppress `printf` calls that do not contain `%n` and whose return value is not used. We cannot really assert nobody has done this, and students ought to learn the provenance of information since an important part of engineering is knowing how you know something (is it specified in a standard, is it just assumed, is it provable, and so on). – Eric Postpischil Jan 15 '19 at 11:34
  • 1
    @EricPostpischil Thanks for that! Very valuable information. – iBug Jan 15 '19 at 11:39
  • 9
    @EricPostpischil See [Damon's answer](https://stackoverflow.com/a/54199638/5958455) for reference from the POSIX standard - supressing a `write(2)` call to write stdout is forbidden by the standard. – iBug Jan 15 '19 at 13:29
  • 4
    AFAIK, console output is pretty slow so the time saved by writting to /dev/null could be significant. It would be less effective if you were writting to file, though. – SJuan76 Jan 15 '19 at 15:12
  • 3
    @SJuan76 Writing to a terminal isn't necessarily slow, *displaying* it is (i.e. depends on terminal). – iBug Jan 15 '19 at 15:13
  • 1
    @SJuan76 There are cases where one would still care about the runtime costs. I've personally worked in situations where I found the need to have a debug print statement running in a tight loop for debugging, where the runtime cost of formatting the output with printf or sprintf would be excessive in a runtime environment. It's not clear if the OP is in such a situation, but it's a case I've found where the formatting costs alone are sufficient to matter. – Cort Ammon Jan 15 '19 at 21:24
  • @iBug So if no terminal physically connected to the system(by wire or tftp, etc..), then print function will have the same affections as redirecting to dev/null? Since they both have no displaying... – Michael Jan 16 '19 at 01:43
  • @MichaelPeng Depends on the handler associated with the terminal - usually it can't be as fast as /dev/null, but some programs create pseudo-terminals (`/dev/pts/x`) that may be faster than other terminals. – iBug Jan 16 '19 at 02:44
  • Who's right? Pretty Much. First name Pretty, second name Much. – Jasper Jan 17 '19 at 14:23
  • LOL pretty joke Jasper @Jasper – Michael Jan 18 '19 at 02:51
  • *In theory, there is no reason a C implementation might not inspect stdout, learn it is /dev/null, and suppress printf calls that do not contain %n and whose return value is not used* -- this is not true, and even if it were, no implementation would do so. It's sad that the comment got 25 upvotes, but that's SO for you. – Jim Balter Jan 18 '19 at 04:34
  • @JimBalter No. That sentence solely means "theoretically it's not forbidden", but I also said "I am unaware if anyone is doing that", – iBug Jan 18 '19 at 04:36
  • And I'm saying that even "theoretically" it isn't true. I wasn't commenting on your answer, but rather EricPostpichii's comment. And "No" to what, exactly? – Jim Balter Jan 18 '19 at 04:37
  • *console output is pretty slow so the time saved by writting to /dev/null could be significant* -- this isn't relevant. The question is about the costs of printf when writing to /dev/null vs. not calling printf at all ... the cost of writing to the console doesn't enter into it. – Jim Balter Jan 18 '19 at 04:38
  • 1
    *Writing to a terminal isn't necessarily slow* -- people are using the word "slow" as if it meant something, and making completely irrelevant references to terminals, displays, consoles, and files. The *question* is whether redirecting stdout to /dev/null is *as fast as* eliding the printfs altogether. The clear answer is "no". – Jim Balter Jan 18 '19 at 04:48
41

The printf function will write to stdout. It is not conforming to optimize for /dev/null. Therefore, you will have the overhead of parsing the format string and evaluating any necessary arguments, and you will have at least one syscall, plus you will copy a buffer to kernel address space (which, compared to the cost of the syscall is neglegible).

This answer is based on the specific documentation of POSIX.

System Interfaces
dprintf, fprintf, printf, snprintf, sprintf - print formatted output

The fprintf() function shall place output on the named output stream. The printf() function shall place output on the standard output stream stdout. The sprintf() function shall place output followed by the null byte, '\0', in consecutive bytes starting at *s; it is the user's responsibility to ensure that enough space is available.

Base Definitions
shall
For an implementation that conforms to POSIX.1-2017, describes a feature or behavior that is mandatory. An application can rely on the existence of the feature or behavior.

Damon
  • 67,688
  • 20
  • 135
  • 185
  • 2
    The kernel is responsible for copying the buffer to kernel address space, and the null driver *probably* omits that step (on Linux, it definitely does - it does not even check that it is a valid address) – Random832 Jan 15 '19 at 15:13
  • 1
    Maybe make it a bit clearer, that not every call to printf will result in a syscall - at the moment this is a bit ambiguous, otherwise great answer. – Voo Jan 15 '19 at 19:46
  • 11
    -1, this ignores the as-if rule (which applies to the *entire implementation*, not just the compiler). If the implementation as a whole can prove that there is no difference in observable behavior, it can make any optimization it likes, even after compilation has completed. Now, if we were talking about a language other than C, you might be right. – Kevin Jan 15 '19 at 19:47
  • 5
    @Kevin Does the as-if rule of the **C** standard apply to the **POSIX** standard too? – Angew is no longer proud of SO Jan 16 '19 at 12:31
  • 1
    @Kevin Since calls to I/O functions are observable behavior themselves, isn't optimization restricted. If the standard said that only I/O is observable effect, then removing printf to /dev/null would be possible. – jinawee Jan 16 '19 at 16:34
  • 1
    @Agnew: `printf` is listed in the POSIX standard as "confirming to the C standard," so yes it does. – Kevin Jan 16 '19 at 17:19
  • 2
    @Kevin: No, it says "aligned with", that's different. On the contrary: Language-Dependent Services for the C Programming Language states: _"Implementors seeking to claim conformance using the ISO C standard shall claim POSIX conformance as described in POSIX Conformance"_. – Damon Jan 16 '19 at 17:36
  • 1
    @jinawee: No, they are not. [Observable behavior](https://en.cppreference.com/w/cpp/language/as_if) consists of 1. accesses of volatile variables, 2. data written to files, 3. data written to "interactive devices" (ttys), and 4. the FPU mode. Data "written" to /dev/null is none of those four things. – Kevin Jan 16 '19 at 18:56
  • 1
    @Kevin Unless there has been some modification wrt. C++ 2003, that's not observable behavior: _The observable behavior of the abstract machine is its sequence of reads and writes to volatile data and calls to library I/O functions._ In fact, "interactive devices" is only mentioned in the requirement _The input and output dynamics of interactive devices shall take place in such a fashion that prompting messages actually appear prior to a program waiting for input. What constitutes an interactive device is implementation-defined._ – jinawee Jan 17 '19 at 10:39
  • @Kevin *-1, this ignores the as-if rule* A bit late, but the as-if rule can not be reliably applied here - and certainly not for an entire long-lived program - because whatever `stdout` happens to be connected to [can change dynamically](https://port70.net/~nsz/c/c11/n1570.html#7.21.5.4). And it is perfectly conforming with all versions of the C standard to **not** perform any eliding under the as-if rule, so POSIX requiring all calls to `printf()` to result in actual writes to `stdout` in no way violates the C standard. – Andrew Henle Apr 17 '23 at 19:13
11

The printf function writes to stdout. If the file descriptor connected to stdout is redirected to /dev/null then no output will be written anywhere (but it will still be written), but the call to printf itself and the formatting it does will still happen.

Some programmer dude
  • 400,186
  • 35
  • 402
  • 621
  • 1
    @OP: Addition: You can further reduce the cost of `printf()` by creating a new driver which provides a new `FILE *` (depending on if your platform supports that). In this case, you can create a data sink which discards the data. The cost for formatting etc. still remains, but the OS call for writing to `/dev/null` goes away. – glglgl Jan 15 '19 at 09:48
  • 2
    Answers like this should state they are based on general understanding of common implementations and not upon specific documentation. In theory, there is no reason a C implementation might not inspect `stdout`, learn it is /dev/null, and suppress `printf` calls that do not contain `%n` and whose return value is not used. We cannot really assert nobody has done this, and students ought to learn the provenance of information since an important part of engineering is knowing how you know something (is it specified in a standard, is it just assumed, is it provable, and so on). – Eric Postpischil Jan 15 '19 at 11:32
5

Write your own that wraps printf() using the printf() source as a guideline, and returning immediately if a noprint flag is set. The downside of this is when actually printing it will consume more resources because of having to parse the format string twice. But it uses negligible resources when not printing. Can't simply replace printf() because the underlying calls inside printf() can change with a newer version of the stdio library.

void printf2(const char *formatstring, ...);

MAXdB
  • 115
  • 2
  • 10
  • 5
    If you use `vprintf` inside `printf2` you will not have to parse the format string twice. – zwol Jan 15 '19 at 21:29
4

Generally speaking, an implementation is permitted to perform such optimisations if they do not affect the observable (functional) outputs of the program. In the case of printf(), that would mean that if the program doesn't use the return value, and if there are no %n conversions, then the implementation would be allowed to do nothing.

In practice, I'm not aware of any implementation on Linux that currently (early 2019) performs such an optimisation - the compilers and libraries I'm familiar with will format the output and write the result to the null device, relying on the kernel' to ignore it.

You may want to write a forwarding function of your own if you really need to save the cost of formatting when the output is not used - you'll want to it to return void, and you should check the format string for %n. (You could use snprintf with a NULL and 0 buffer if you need those side-effects, but the savings are unlikely to repay the effort invested).

Toby Speight
  • 27,591
  • 48
  • 66
  • 103
  • Since the observable behavior of the abstract machine includes calls to library I/O functions, surely `printf` at least it would have to be called right? Although I'm not sure how two different calls to a library function are considered different or not... – jinawee Jan 16 '19 at 16:29
1

in C writing 0; does execute and nothing, which is similar to ;.

means you can write a macro like

#if DEBUG
#define devlognix(frmt,...) fprintf(stderr,(frmt).UTF8String,##__VA_ARGS__)
#else
#define nadanix 0
#define devlognix(frmt,...) nadanix
#endif
#define XYZKitLogError(frmt, ...) devlognix(frmt)

where XYZKitLogError would be your Log command. or even

#define nadanix ;

which will kick out all log calls at compile time and replace with 0; or ; so it gets parsed out.

you will get Unused variable warnings, but it does what you want and this side effect can even be helpful because it tells you about computation that is not needed in your release.

.UTF8String is an Objective-C method converting NSStrings to const char* - you don't need.

Ol Sen
  • 3,163
  • 2
  • 21
  • 30