Redirection of stdout to a file not working

Question

I have a script that uses subprocesses to fetch HTML:

misha@misha-K42Jr:~/git/domain_classifier$ python webkit_retrieve.py error-cut.txt html/error -N 5
http://kurabo.co.jp HostNotFoundError
http://monarch.com HostNotFoundError
http://nssmgmt.com HostNotFoundError
http://sbcglobal.net HostNotFoundError
http://dynamixcorp.com SslHandshakeFailedError
http://groupe-synox.com RemoteHostClosedError
QFont::setPixelSize: Pixel size <= 0 (0)
http://www.cnn.com NoError
http://pacbell.net TimeoutError

If I run the same script, but redirect output to a file, I get nothing in the output:

misha@misha-K42Jr:~/git/domain_classifier$ python webkit_retrieve.py error-cut.txt html/error -N 5 > stdout.txt
QFont::setPixelSize: Pixel size <= 0 (0)
misha@misha-K42Jr:~/git/domain_classifier$ cat stdout.txt
misha@misha-K42Jr:~/git/domain_classifier$

Why is the output empty? Should it not contain the same things that were printed to stdout in the first case?

The question is not about merge stdout and stderr but why redirected stdout produce an empty file

The file is empty, hence `cat stdout.txt` prints absolutely nothing. — mpenkov, Nov 20 '12 at 04:20
What is the python source for setting up the output file handles and doing the output itself? — Richard Chambers, Dec 03 '21 at 13:47
If the output for both commands is **different** and `stdout.txt` is really empty then the python script acts differently if the stdout is redirected — Diego Torres Milano, Dec 04 '21 at 00:08

score 35 · Answer 1 · edited Nov 20 '12 at 08:38

35

use &> for redirection, this should redirect stdout and stderr to designated file

edited Nov 20 '12 at 08:38

Kazekage Gaara

14,972
14
61
108

answered Nov 20 '12 at 08:18

Saddam Abu Ghaida

6,381
2
22
29

5

I've been using `>` successfully forever and after 20 years it stopped (not sure if it has anything to do with the fact it's a python script that wasn't working). Now with the `&>` it worked. What's that all about? – cristianoms Feb 19 '20 at 15:54

score 1 · Answer 2 · answered Nov 20 '12 at 03:59

1

You have sent stdout to the file, but your program is reporting errors which go to stderr. To setup redirection of stderr, use 2> syntax.

This link might help: http://www.tldp.org/LDP/abs/html/io-redirection.html

answered Nov 20 '12 at 03:59

mcalex

6,628
5
50
80

If it was echoing to `stderr`, wouldn't it still show at the terminal when the command is run with just `stdout` redirected? – mgarciaisaia Nov 20 '12 at 04:00
good point. It almost looks like stderr is actually being redirected, given the 'Pixel size' line's appearance – mcalex Nov 20 '12 at 04:08
stderr is correctly being printed to standard output (the pixel size stuff) as per the default behavior on linux systems. I don't want to redirect stderr, I want to redirect stdout. – mpenkov Nov 20 '12 at 04:21

score 0 · Answer 3 · answered Jul 14 '22 at 02:12

0

I think I was looking for solution a couple of moment ago.

Your best bet is to run something like this:

yourcommand | cat > redirectfile

answered Jul 14 '22 at 02:12

Nurain150

1

...what is `cat` doing here? Wouldn't `yourcommand > redirectfile` do exactly the same thing? – ChrisGPT was on strike Jul 14 '22 at 22:18

score -2 · Answer 4 · answered Nov 20 '12 at 04:00

-2

Your output is going to stderror. Try redirecting both stderror and stdout using the following:

<command> 2>&1 <file>

answered Nov 20 '12 at 04:00

David

6,462
2
25
22

2

If the output was going to stderror, then I would see it in the console (as I do in the first case). In the second case, the output is being redirected to a file, but the file ends up empty. – mpenkov Nov 20 '12 at 04:22
Doesn't work, file is not created – m0skit0 Sep 08 '19 at 22:58

Redirection of stdout to a file not working

4 Answers4