How to handle both `with open(...)` and `sys.stdout` nicely?

Question

Often I need to output data either to file or, if file is not specified, to stdout. I use the following snippet:

if target:
    with open(target, 'w') as h:
        h.write(content)
else:
    sys.stdout.write(content)

I would like to rewrite it and handle both targets uniformly.

In ideal case it would be:

with open(target, 'w') as h:
    h.write(content)

but this will not work well because sys.stdout is be closed when leaving with block and I don't want that. I neither want to

stdout = open(target, 'w')
...

because I would need to remember to restore original stdout.

Related:

• Redirect stdout to a file in Python?
• Handling Exceptions - interesting article about handling exceptions in Python, as compared to C++

Edit

I know that I can wrap target, define separate function or use context manager. I look for a simple, elegant, idiomatic solution fitting that wouldn't require more than 5 lines

Answer 1

Just thinking outside of the box here, how about a custom open() method?

import sys
import contextlib

@contextlib.contextmanager
def smart_open(filename=None):
    if filename and filename != '-':
        fh = open(filename, 'w')
    else:
        fh = sys.stdout

    try:
        yield fh
    finally:
        if fh is not sys.stdout:
            fh.close()

Use it like this:

# For Python 2 you need this line
from __future__ import print_function

# writes to some_file
with smart_open('some_file') as fh:
    print('some output', file=fh)

# writes to stdout
with smart_open() as fh:
    print('some output', file=fh)

# writes to stdout
with smart_open('-') as fh:
    print('some output', file=fh)

Answer 2

Stick with your current code. It's simple and you can tell exactly what it's doing just by glancing at it.

Another way would be with an inline if:

handle = open(target, 'w') if target else sys.stdout
handle.write(content)

if handle is not sys.stdout:
    handle.close()

But that isn't much shorter than what you have and it looks arguably worse.

You could also make sys.stdout unclosable, but that doesn't seem too Pythonic:

sys.stdout.close = lambda: None

with (open(target, 'w') if target else sys.stdout) as handle:
    handle.write(content)

Answer 3

An improvement of Wolph's answer

import sys
import contextlib

@contextlib.contextmanager
def smart_open(filename: str, mode: str = 'r', *args, **kwargs):
    '''Open files and i/o streams transparently.'''
    if filename == '-':
        if 'r' in mode:
            stream = sys.stdin
        else:
            stream = sys.stdout
        if 'b' in mode:
            fh = stream.buffer  # type: IO
        else:
            fh = stream
        close = False
    else:
        fh = open(filename, mode, *args, **kwargs)
        close = True

    try:
        yield fh
    finally:
        if close:
            try:
                fh.close()
            except AttributeError:
                pass

This allows binary IO and pass eventual extraneous arguments to open if filename is indeed a file name.

Answer 4

Why LBYL when you can EAFP?

try:
    with open(target, 'w') as h:
        h.write(content)
except TypeError:
    sys.stdout.write(content)

Why rewrite it to use the with/as block uniformly when you have to make it work in a convoluted way? You'll add more lines and reduce performance.

Answer 5

As pointed in Conditional with statement in Python, Python 3.7 allows using contextlib.nullcontext for that:

from contextlib import nullcontext

with open(target, "w") if target else nullcontext(sys.stdout) as f:
    f.write(content)

Answer 6

Another possible solution: do not try to avoid the context manager exit method, just duplicate stdout.

with (os.fdopen(os.dup(sys.stdout.fileno()), 'w')
      if target == '-'
      else open(target, 'w')) as f:
      f.write("Foo")

Answer 7

If it's fine that sys.stdout is closed after with body, you can also use patterns like this:

# Use stdout when target is "-"
with open(target, "w") if target != "-" else sys.stdout as f:
    f.write("hello world")

# Use stdout when target is falsy (None, empty string, ...)
with open(target, "w") if target else sys.stdout as f:
    f.write("hello world")

or even more generally:

with target if isinstance(target, io.IOBase) else open(target, "w") as f:
    f.write("hello world")

Answer 8

import contextlib
import sys

with contextlib.ExitStack() as stack:
    h = stack.enter_context(open(target, 'w')) if target else sys.stdout
    h.write(content)

Just two extra lines if you're using Python 3.3 or higher: one line for the extra import and one line for the stack.enter_context.

Answer 9

I'd also go for a simple wrapper function, which can be pretty simple if you can ignore the mode (and consequently stdin vs. stdout), for example:

from contextlib import contextmanager
import sys

@contextmanager
def open_or_stdout(filename):
    if filename != '-':
        with open(filename, 'w') as f:
            yield f
    else:
        yield sys.stdout

Answer 10

Okay, if we are getting into one-liner wars, here's:

(target and open(target, 'w') or sys.stdout).write(content)

I like Jacob's original example as long as context is only written in one place. It would be a problem if you end up re-opening the file for many writes. I think I would just make the decision once at the top of the script and let the system close the file on exit:

output = target and open(target, 'w') or sys.stdout
...
output.write('thing one\n')
...
output.write('thing two\n')

You could include your own exit handler if you think its more tidy

import atexit

def cleanup_output():
    global output
    if output is not sys.stdout:
        output.close()

atexit(cleanup_output)

Answer 11

This is a simpler and shorter version of the accepted answer

import contextlib, sys


def writer(fn): 
    @contextlib.contextmanager
    def stdout():
        yield sys.stdout
    return open(fn, 'w') if fn else stdout()

usage:

with writer('') as w:
    w.write('hello\n')

with writer('file.txt') as w:
    w.write('hello\n')

Answer 12

If you really must insist on something more "elegant", i.e. a one-liner:

>>> import sys
>>> target = "foo.txt"
>>> content = "foo"
>>> (lambda target, content: (lambda target, content: filter(lambda h: not h.write(content), (target,))[0].close())(open(target, 'w'), content) if target else sys.stdout.write(content))(target, content)

foo.txt appears and contains the text foo.

Answer 13

How about opening a new fd for sys.stdout? This way you won't have any problems closing it:

if not target:
    target = "/dev/stdout"
with open(target, 'w') as f:
    f.write(content)

Answer 14

if (out != sys.stdout):
    with open(out, 'wb') as f:
        f.write(data)
else:
    out.write(data)

Slight improvement in some cases.

Answer 15

The following solution is not a beauty, but from a time long, long ago; just before with ...

handler = open(path, mode = 'a') if path else sys.stdout
try:
    print('stuff', file = handler)
    ... # other stuff or more writes/prints, etc.
except Exception as e:
    if not (path is None): handler.close()
    raise e
handler.close()

Answer 16

One way to solve it is with polymorphism. Pathlib.path has an open method that functions as you would expect:

from pathlib import Path

output = Path("/path/to/file.csv")

with output.open(mode="w", encoding="utf-8") as f:
    print("hello world", file=f)

we can copy this interface for printing

import sys

class Stdout:
    def __init__(self, *args):
        pass

    def open(self, mode=None, encoding=None):
        return self

    def __enter__(self):
        return sys.stdout

    def __exit__(self, exc_type, exc_value, traceback):
        pass

Now we simply replace Path with Stdout

output = Stdout("/path/to/file.csv")

with output.open(mode="w", encoding="utf-8") as f:
    print("hello world", file=f)

This isn't necessarily better than overloading open, but it's a convenient solution if you're using Path objects.

Answer 17

With python 3 you can used wrap stdout file descriptor with IO object and avoid closing on context leave it with closefd=False:

h = open(target, 'w') if target else open(sys.stdout.fileno(), 'w', closefd=False)

with h as h:
    h.write(content)