3

There was a redirect_output function in IPython.utils, and there was a %%capture magic function, but these are now gone, and this thread on the topic is now outdated.

I'd like to do something like the following:

from IPython.utils import io
from __future__ import print_function
with io.redirect_output(stdout=False, stderr="stderr_test.txt"):
    while True:
        print('hello!', file=sys.stderr)

Thoughts? For more context, I am trying to capture the output of some ML functions that run for hours or days, and output a line every 5-10 seconds to stderr. I then want to take the output, munge it, and plot the data.

Community
  • 1
  • 1
paulperry
  • 826
  • 8
  • 16

2 Answers2

3

You could probably try replacing sys.stderr with some other file descriptor the same way as suggested here.

import sys
oldstderr = sys.stderr
sys.stderr = open('log.txt', 'w')
# do something
sys.stderr = oldstderr

Update: starting form Python 3.4, you should consuder using contextlib.redirect_stdout() instead, like this:

f = io.StringIO()
with redirect_stdout(f):
    print('a')
s = f.getvalue()
Ben Usman
  • 7,969
  • 6
  • 46
  • 66
1

@Ben, just replacing sys.stderr did not work, and the full flush logic suggested in the post was necessary. But thank you for the pointer as it finally gave me a working version:

import sys
oldstderr = sys.stderr
sys.stderr = open('log.txt', 'w')
class flushfile():
    def __init__(self, f):
        self.f = f
    def __getattr__(self,name): 
        return object.__getattribute__(self.f, name)
    def write(self, x):
        self.f.write(x)
        self.f.flush()
    def flush(self):
        self.f.flush()
sys.sterr = flushfile(sys.stderr)
from __future__ import print_function
# some long running function here, e.g. 
for i in range(1000000):
    print('hello!', file=sys.stderr)
sys.stderr = oldstderr

It would have been nice if Jupyter kept the redirect_output() function and/or the %%capture magic.

Community
  • 1
  • 1
paulperry
  • 826
  • 8
  • 16
  • This works (but I do feel there ought to be a more concise "built-in" way). To get all output, calling `sys.stderr.flush()` is necessary before switching back to `oldstderr`. – Dmitri Jun 28 '16 at 15:49
  • `_io.TextIOWrapper` already provides `write()` and `flush()`. Not sure why the wrapper class is needed. – Dmitri Jun 28 '16 at 16:20