Blocking sys.stdout and stderr does not prevent C code from printing

Question

I am including in my python code a function compiled in c via a cython wrapper. I have to take that function as given and cannot change it. Unfortunately, when I run that function, I see output that is bothering me.

I have tried a lot of tricks that are supposed to get rid of it, all of which play with sys.stdout or sys.stderr -- most noteably, the new contextlib.redirect_stdout. However, nothing I tried managed to silence the output.

At the most basic level, I simply tried setting

    sys.stdout = open(os.devnull, 'w')
    sys.stderr = open(os.devnull, 'w')

Which is not a safe, or practicable way of doing it, but it should shut the function up. Unfortunately, I can still see the output. What am I missing? Is there perhaps another "output type" besides stdout that this function might be using?

If it helps, I am inside a Pycharm debugging session and see this output in my debugging console.

Updated question to reflect that changing stderr did not help

The `contextlib` module provides a `redirect_stdout` context manager which lets you *temporarily* redirect to `os.devnull` like you are doing. It also provides`redirect_stderr` if that's the file being written to. — chepner, Aug 09 '22 at 12:30
you probably have to call another C function beforehand to close stdout. python stdout won't connect with generated code. — Jean-François Fabre, Aug 09 '22 at 12:32
@deceze I tried shutting down stderr in a similar fashion (see update), function is still creating output as if its life depended on it. — FooBar, Aug 09 '22 at 12:38
I think it may be printing to the C stout/stderr (which aren't the same as the Python ones) (such is essentially what Jean-FrançoisFabre said earlier I think) — DavidW, Aug 09 '22 at 12:39
@DavidW They're the same, but it's also possible that the code is writing directly to the terminal. — chepner, Aug 09 '22 at 12:41
@chepner If I understand right, the Python `sys.stdout`/`sys.stderr` wraps the C ones. So modifying the C ones will likely interfere with the Python but I don't think it'll work the other way around. I may not be right though - I try my best not to deal with this stuff if possible! — DavidW, Aug 09 '22 at 12:44
Possibly a duplicate of https://stackoverflow.com/a/63328631/4657412? — DavidW, Aug 09 '22 at 12:51
@DavidW You are correct. If you have a Python script that loads a C shared object but then hijacks `sys.stdout`/`stderr`, any stdout/stderr output from the C code will still get through. — 0x5453, Aug 09 '22 at 12:51
OK, I see the distinction being made: there's file descriptor 1 (standard output, which is the same throughout the process, whether it's the Python interpreter or any shared libraries it links to), and then there's `sys.stdout` which is a Python object *initialized* to the same file handle but can be made to wrap a different handle. — chepner, Aug 09 '22 at 13:21

VPfB · Answer 1 · 2022-08-09T15:43:49.570

A C function prints to a file descriptor (1 for stdout, 2 for stderr). If you want to prevent the printing, redirect that FD, that can be done also temporarily. Here is a litte demo:

import os

STDOUT = 1 

saved_fd = os.dup(STDOUT)
null_fd = os.open(os.devnull, os.O_WRONLY)
os.dup2(null_fd, STDOUT)
os.system('echo TEST 1')    # redirected to /dev/null
os.dup2(saved_fd, STDOUT)
os.system('echo TEST 2')    # normal

# note: close the null_fd, saved_fd when no longer needed

If the C code opens the terminal device itself, there is very little you can do to prevent it. But that would be very unusual (I would even say a bug) unless there is a specific reason to do so.

score 0 · Answer 2 · answered Aug 09 '22 at 12:30

Is there perhaps another "output type" besides stdout that this function might be using?

Yes, there exist stderr, which would be unaffected by stdout redirect, simple example, let printer.py content be

import sys
sys.stderr.write("printing to stderr")

then running in terminal

python printer.py > output.txt

lead to appearance of

printing to stderr

as > output.txt redirects only stdout.

Blocking sys.stdout and stderr does not prevent C code from printing

2 Answers2

Linked