182

I'm using python's ftplib to write a small FTP client, but some of the functions in the package don't return string output, but print to stdout. I want to redirect stdout to an object which I'll be able to read the output from.

I know stdout can be redirected into any regular file with:

stdout = open("file", "a")

But I prefer a method that doesn't uses the local drive.

I'm looking for something like the BufferedReader in Java that can be used to wrap a buffer into a stream.

martineau
  • 119,623
  • 25
  • 170
  • 301
Avihu Turzion
  • 3,284
  • 4
  • 25
  • 34

9 Answers9

251
from cStringIO import StringIO # Python3 use: from io import StringIO
import sys

old_stdout = sys.stdout
sys.stdout = mystdout = StringIO()

# blah blah lots of code ...

sys.stdout = old_stdout

# examine mystdout.getvalue()
Aziz Alto
  • 19,057
  • 5
  • 77
  • 60
Ned Batchelder
  • 364,293
  • 75
  • 561
  • 662
  • 69
    +1, you don't need to keep a reference to the original `stdout` object, as it is always available at `sys.__stdout__`. See http://docs.python.org/library/sys.html#sys.__stdout__. – Ayman Hourieh Aug 02 '09 at 14:00
  • 104
    Well, that's an interesting debate. The absolute original stdout is available, but when replacing like this, it's better to use an explicit save as I've done, since someone else could have replaced stdout and if you use __stdout__, you'd clobber their replacement. – Ned Batchelder Aug 02 '09 at 14:25
  • 7
    would this operation in one thread alter the behavior of other threads? I mean is it threadsafe? – Anuvrat Parashar Sep 13 '12 at 11:19
  • @AnuvratParashar: I think that would be an excellent question to ask on its own. – Dennis Williamson Nov 27 '12 at 19:47
  • @AnuvratParashar: I'm pretty sure it is not thread safe, unless there are details I don't know about (or didn't notice), I used this method to retrieve `print` calls in other threads. – PhilMacKay Jul 23 '13 at 14:03
  • 1
    This doesn't work if `sys.stdout.buffer` (Python 3) is used. See [my answer](http://stackoverflow.com/a/19345047/1904815) for a solution working in that case. For old Python-2-only code this is still the best solution. – JonnyJD Oct 13 '13 at 12:12
  • 9
    I highly recommend to reassign the old stdout in a `finally:` block, so it is also reassigned if an exception is risen in between. `try: bkp = sys.stdout ... ... finally: sys.stdout = bkp` – Matthias Kuhn Feb 20 '14 at 13:37
  • @erikb85: `subprocess.call()` requires that `sys.stdout.fileno()` is redirected, the solution that just replaces `sys.stdout` won't work even for functions that store `sys.stdout` value locally in Python. See [`stdout_redirected()` that redirects `sys.stdout.fileno()` instead of replacing `sys.stdout`](http://stackoverflow.com/a/22434262/4279). Though if it is your code, you could use `subprocess.call(stdout=..)` to redirect stdout of a subprocess (or just call `subprocess.check_output()`). – jfs Mar 31 '14 at 12:11
  • 25
    If you want to use this in Python 3, replace cStringIO with io . – Anthony Labarre Nov 29 '14 at 11:41
  • for python3 use from io import StringIO – karansthr Feb 28 '18 at 16:49
  • +1 for @NedBatchelder comment to store stdout. E.g. in **Jupyter notebooks**, stdout is altered and resetting to `sys.__stdout__` screws up printing in Jupyter. – jrieke Sep 08 '18 at 22:41
  • Replacing built-in library attributes is bad practice (a.o. because of thread safety) – Martin Apr 10 '19 at 15:39
144

There is a contextlib.redirect_stdout() function in Python 3.4+:

import io
from contextlib import redirect_stdout

with io.StringIO() as buf, redirect_stdout(buf):
    print('redirected')
    output = buf.getvalue()

Here's a code example that shows how to implement it on older Python versions.

Boris Verkhovskiy
  • 14,854
  • 11
  • 100
  • 103
jfs
  • 399,953
  • 195
  • 994
  • 1,670
41

Just to add to Ned's answer above: you can use this to redirect output to any object that implements a write(str) method.

This can be used to good effect to "catch" stdout output in a GUI application.

Here's a silly example in PyQt:

import sys
from PyQt4 import QtGui

class OutputWindow(QtGui.QPlainTextEdit):
    def write(self, txt):
        self.appendPlainText(str(txt))

app = QtGui.QApplication(sys.argv)
out = OutputWindow()
sys.stdout=out
out.show()
print "hello world !"
Jakub M.
  • 32,471
  • 48
  • 110
  • 179
Nicolas Lefebvre
  • 4,247
  • 1
  • 25
  • 29
11

A context manager for python3:

import sys
from io import StringIO


class RedirectedStdout:
    def __init__(self):
        self._stdout = None
        self._string_io = None

    def __enter__(self):
        self._stdout = sys.stdout
        sys.stdout = self._string_io = StringIO()
        return self

    def __exit__(self, type, value, traceback):
        sys.stdout = self._stdout

    def __str__(self):
        return self._string_io.getvalue()

use like this:

>>> with RedirectedStdout() as out:
>>>     print('asdf')
>>>     s = str(out)
>>>     print('bsdf')
>>> print(s, out)
'asdf\n' 'asdf\nbsdf\n'
Bob
  • 5,809
  • 5
  • 36
  • 53
9

Starting with Python 2.6 you can use anything implementing the io.TextIOBase API as a replacement. This solution also enables you to use sys.stdout.buffer.write() in Python 3 to write (already) encoded byte strings to standard output (see stdout in Python 3). Using io.StringIO wouldn't work then, because neither sys.stdout.buffer nor sys.stdout.encoding would be available.

A solution using io.TextIOWrapper:

import io
import sys

# Setup stdout.
old_stdout = sys.stdout
sys.stdout = io.TextIOWrapper(io.BytesIO(), sys.stdout.encoding)

# Write to stdout or stdout.buffer.
...

# Read from stdout.
sys.stdout.seek(0)
out = sys.stdout.read()

# Restore stdout.
sys.stdout.close()
sys.stdout = old_stdout

This solution works for Python 2 >= 2.6 and Python 3.

Please note that the new sys.stdout.write() only accepts unicode strings and sys.stdout.buffer.write() only accepts byte strings. Code that is built to run on Python 2 and 3 without changes often makes use of sys.stdout.buffer.

So to have sys.stdout.write() accept both unicode and byte strings, you can instead use this io.TextIOWrapper subclass:

class StdoutBuffer(io.TextIOWrapper):

    def write(self, string):
        try:
            return super(StdoutBuffer, self).write(string)
        except TypeError:
            # Redirect encoded byte strings directly to buffer.
            return super(StdoutBuffer, self).buffer.write(string)

You don't have to set the encoding of the buffer the sys.stdout.encoding, but this helps when using this method for testing/comparing script output.

Géry Ogam
  • 6,336
  • 4
  • 38
  • 67
JonnyJD
  • 2,593
  • 1
  • 28
  • 44
  • This answer helped me when setting up an Environment object's stdout param for use with Httpie's core.py. – fragorl Dec 06 '17 at 23:52
8

This method restores sys.stdout even if there's an exception. It also gets any output before the exception.

import io
import sys

real_stdout = sys.stdout
fake_stdout = io.BytesIO()   # or perhaps io.StringIO()
try:
    sys.stdout = fake_stdout
    # do what you have to do to create some output
finally:
    sys.stdout = real_stdout
    output_string = fake_stdout.getvalue()
    fake_stdout.close()
    # do what you want with the output_string

Tested in Python 2.7.10 using io.BytesIO()

Tested in Python 3.6.4 using io.StringIO()

Bob, added for a case if you feel anything from the modified / extended code experimentation might get interesting in any sense, otherwise feel free to delete it

Ad informandum ... a few remarks from extended experimentation during finding some viable mechanics to "grab" outputs, directed by numexpr.print_versions() directly to the <stdout> ( upon a need to clean GUI and collecting details into debugging-report )

# THIS WORKS AS HELL: as Bob Stein proposed years ago:
#  py2 SURPRISEDaBIT:
#
import io
import sys
#
real_stdout = sys.stdout                        #           PUSH <stdout> ( store to REAL_ )
fake_stdout = io.BytesIO()                      #           .DEF FAKE_
try:                                            # FUSED .TRY:
    sys.stdout.flush()                          #           .flush() before
    sys.stdout = fake_stdout                    #           .SET <stdout> to use FAKE_
    # ----------------------------------------- #           +    do what you gotta do to create some output
    print 123456789                             #           + 
    import  numexpr                             #           + 
    QuantFX.numexpr.__version__                 #           + [3] via fake_stdout re-assignment, as was bufferred + "late" deferred .get_value()-read into print, to finally reach -> real_stdout
    QuantFX.numexpr.print_versions()            #           + [4] via fake_stdout re-assignment, as was bufferred + "late" deferred .get_value()-read into print, to finally reach -> real_stdout
    _ = os.system( 'echo os.system() redir-ed' )#           + [1] via real_stdout                                 + "late" deferred .get_value()-read into print, to finally reach -> real_stdout, if not ( _ = )-caught from RET-d "byteswritten" / avoided from being injected int fake_stdout
    _ = os.write(  sys.stderr.fileno(),         #           + [2] via      stderr                                 + "late" deferred .get_value()-read into print, to finally reach -> real_stdout, if not ( _ = )-caught from RET-d "byteswritten" / avoided from being injected int fake_stdout
                       b'os.write()  redir-ed' )#  *OTHERWISE, if via fake_stdout, EXC <_io.BytesIO object at 0x02C0BB10> Traceback (most recent call last):
    # ----------------------------------------- #           ?                              io.UnsupportedOperation: fileno
    #'''                                                    ? YET:        <_io.BytesIO object at 0x02C0BB10> has a .fileno() method listed
    #>>> 'fileno' in dir( sys.stdout )       -> True        ? HAS IT ADVERTISED,
    #>>> pass;            sys.stdout.fileno  -> <built-in method fileno of _io.BytesIO object at 0x02C0BB10>
    #>>> pass;            sys.stdout.fileno()-> Traceback (most recent call last):
    #                                             File "<stdin>", line 1, in <module>
    #                                           io.UnsupportedOperation: fileno
    #                                                       ? BUT REFUSES TO USE IT
    #'''
finally:                                        # == FINALLY:
    sys.stdout.flush()                          #           .flush() before ret'd back REAL_
    sys.stdout = real_stdout                    #           .SET <stdout> to use POP'd REAL_
    sys.stdout.flush()                          #           .flush() after  ret'd back REAL_
    out_string = fake_stdout.getvalue()         #           .GET string           from FAKE_
    fake_stdout.close()                         #                <FD>.close()
    # +++++++++++++++++++++++++++++++++++++     # do what you want with the out_string
    #
    print "\n{0:}\n{1:}{0:}".format( 60 * "/\\",# "LATE" deferred print the out_string at the very end reached -> real_stdout
                                     out_string #                   
                                     )
'''
PASS'd:::::
...
os.system() redir-ed
os.write()  redir-ed
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
123456789
'2.5'
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Numexpr version:   2.5
NumPy version:     1.10.4
Python version:    2.7.13 |Anaconda 4.0.0 (32-bit)| (default, May 11 2017, 14:07:41) [MSC v.1500 32 bit (Intel)]
AMD/Intel CPU?     True
VML available?     True
VML/MKL version:   Intel(R) Math Kernel Library Version 11.3.1 Product Build 20151021 for 32-bit applications
Number of threads used by default: 4 (out of 4 detected cores)
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
>>>

EXC'd :::::
...
os.system() redir-ed
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
123456789
'2.5'
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Numexpr version:   2.5
NumPy version:     1.10.4
Python version:    2.7.13 |Anaconda 4.0.0 (32-bit)| (default, May 11 2017, 14:07:41) [MSC v.1500 32 bit (Intel)]
AMD/Intel CPU?     True
VML available?     True
VML/MKL version:   Intel(R) Math Kernel Library Version 11.3.1 Product Build 20151021 for 32-bit applications
Number of threads used by default: 4 (out of 4 detected cores)
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\

Traceback (most recent call last):
  File "<stdin>", line 9, in <module>
io.UnsupportedOperation: fileno
'''
user3666197
  • 1
  • 6
  • 50
  • 92
Bob Stein
  • 16,271
  • 10
  • 88
  • 101
5

In Python3.6, the StringIO and cStringIO modules are gone, you should use io.StringIO instead.So you should do this like the first answer:

import sys
from io import StringIO

old_stdout = sys.stdout
old_stderr = sys.stderr
my_stdout = sys.stdout = StringIO()
my_stderr = sys.stderr = StringIO()

# blah blah lots of code ...

sys.stdout = self.old_stdout
sys.stderr = self.old_stderr

// if you want to see the value of redirect output, be sure the std output is turn back
print(my_stdout.getvalue())
print(my_stderr.getvalue())

my_stdout.close()
my_stderr.close()
haofly
  • 829
  • 12
  • 15
  • 1
    You could improve the quality of your Answer by explaining how the above code works and how this is an improvement over the Questioner's situation. – toonice Apr 09 '17 at 02:28
3

Use pipe() and write to the appropriate file descriptor.

https://docs.python.org/library/os.html#file-descriptor-operations

twasbrillig
  • 17,084
  • 9
  • 43
  • 67
Meredith L. Patterson
  • 4,853
  • 29
  • 30
3

Here's another take on this. contextlib.redirect_stdout with io.StringIO() as documented is great, but it's still a bit verbose for every day use. Here's how to make it a one-liner by subclassing contextlib.redirect_stdout:

import sys
import io
from contextlib import redirect_stdout

class capture(redirect_stdout):

    def __init__(self):
        self.f = io.StringIO()
        self._new_target = self.f
        self._old_targets = []  # verbatim from parent class

    def __enter__(self):
        self._old_targets.append(getattr(sys, self._stream))  # verbatim from parent class
        setattr(sys, self._stream, self._new_target)  # verbatim from parent class
        return self  # instead of self._new_target in the parent class

    def __repr__(self):
        return self.f.getvalue()

Since __enter__ returns self, you have the context manager object available after the with block exits. Moreover, thanks to the __repr__ method, the string representation of the context manager object is, in fact, stdout. So now you have,

with capture() as message:
    print('Hello World!')
print(str(message)=='Hello World!\n')  # returns True
pandichef
  • 706
  • 9
  • 11