When to use sys.stdout instead of sys.stderr?

Question

My boss told me to look at some old code where everything is being sent to stderr. I know that stderr should have the warnings and errors, but when should they really should go to stdout instead?

This program is a service. Some of the messages that it sends to stderr are:

if pid is None:
    message = "pidfile {0} is not running. \n"
    sys.stderr.write(message.format(self.pidfile))

So this one is an output, right? A message about being successful or not should go to stdout, because it's not an error?

And then I got another one that is an exception:

except OSError as err:
    sys.stderr.write('fork #1 failed: {0}\n'.format(err))
    sys.exit(1)

This one as an exception, so it should go to stderr, because it's an error? Or should it go to stdout, because it is a output message just saying that the program failed?

I know what stdout and stderr are --- I read quite a bit before asking. My problem here is being able to identify which messages should go to stderr --- just fatal errors, or any kind of error message?

Both the examples are error messages, but one is just a "Program is not running". I'm having trouble choosing where that kind of message should be sent.

Possible duplicate of [confused about stdin, stdout and stderr?](http://stackoverflow.com/questions/3385201/confused-about-stdin-stdout-and-stderr) — wpercy, Feb 17 '16 at 16:03
Fatal errors should certainly go to stderr. Normal output should go to stdout. There is a gray area for informative messages, but in general they should not go to stdout if they would render the output unusable. People will often direct stdout to a file and stderr to either the terminal or a log file. So you need to decide what division makes the most sense. — Tom Karzes, Feb 17 '16 at 16:04
@TomKarzes: post that as an answer. More informative than the dup Q. — Karoly Horvath, Feb 17 '16 at 16:09
Think of standard output as being what your program *does*, and standard error as being *how* your program does it. — chepner, Feb 17 '16 at 16:21
The actual data that the program outputs should go to stdout, _all_ metadata like error messages and progress reports should go to stderr. So "fork #1 failed" should go to stderr, and so should something like "fork #1 succeeded". The "pidfile {0} is not running" should also go to stderr. — PM 2Ring, Feb 17 '16 at 16:45
Stdout is content that goes through pipelines -- it's the actual data you're generating. Stderr is content that isn't data you're generating, but is *about* how you're generating data. That doesn't mean it has to be an error; **anything that isn't actually output data** belongs on stderr. — Charles Duffy, Feb 18 '16 at 16:04

score 8 · Accepted Answer · edited Feb 17 '16 at 16:31

8

Fatal errors should certainly go to stderr. Normal output should go to stdout.

There is a gray area for informative messages, but in general they should not go to stdout if they would render the output unusable. People will often direct stdout to a file and stderr to either the terminal or a log file.

You need to decide what division makes the most sense.

edited Feb 17 '16 at 16:31

Karoly Horvath

94,607
11
117
176

answered Feb 17 '16 at 16:11

Tom Karzes

22,815
2
22
41

4

I like to think of it this way: standard output is for what your program *does*, and standard error is for *how* your program does it. – chepner Feb 17 '16 at 16:25
Got it. The first one is just an output saying that is not running so is not an error message. The other as a fatal should go to stderr. – DKage Feb 17 '16 at 16:44
@chepner that was very helpful. Going to try to use that as a method to the other ones here. Thanks guys. – DKage Feb 17 '16 at 16:50
3

@Dkage, stderr **is not limited to error messages**; it's for metadata in general. Anything that is informative rather than being actual output data (that you'd want to parse in another program that's reading your generated content) belongs on stderr. – Charles Duffy Feb 18 '16 at 16:05
Can you provide a source for that last claim @CharlesDuffy? I mean, that in python it is also intended for that? I think this will answer my question in https://stackoverflow.com/questions/75580592/why-is-tqdm-output-directed-to-sys-stderr-and-not-to-sys-stdout – Heberto Mayorquin Feb 27 '23 at 12:41
1

@HebertoMayorquin, the convention for stdout and stderr streams is defined by the operating system, not by any specific programming language: it's OS conventions that define how applications running on that OS are supposed to behave, and your program is running on UNIX and expected to honor UNIX conventions no matter what language it's written in. See section 2.5 in https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html (I used the word "informative" above, but the formally-defined word is "diagnostic"). – Charles Duffy Feb 27 '23 at 14:08

Kevin J. Chase · Answer 2 · 2016-02-18T19:29:25.777

Tom Karzes has already given you a correct answer, but to state a more extreme position...

I find it helps to think of stdout and stderr as "standard output" and "standard not-output". You said you thought a message "should go to stdout because it's not an error", but that's backward: stdout is the stream with the high barrier to entry, not stderr.

A message should go to stderr (or a log file, or anything but stdout) unless it is provably part of the desired output --- the final product that the user actually wants. If you can't justify its inclusion with a sentence that starts, "The user asked for...", don't pollute stdout with it. (The next process in the pipeline will thank you.)

Odds are, your users didn't run your program because they wanted to watch a twirling baton, or to read a dozen "Attempting to reconnect..." messages, or even to learn which forks had failed, so those should not be on stdout.

Finally, some messages shouldn't go to either output stream. stderr is presumably being read by a user, so try to limit its output to things that user cares about right now. Keep it free of "situation normal" messages, unless the user has specifically requested them by turning up the verbosity (usually --verbose or --debug). At least allow the user to silence non-errors with a --quiet or --silent option. Most "informational" messages should be sent to a log file instead of the console.

When to use sys.stdout instead of sys.stderr?

2 Answers2

Linked