Why does my revectored sys.stdout.write() method get called after I've unlinked it?

Question

I wrote a tee() class, which redirects stdout to both the terminal and a file, based on Triptych's answer at https://stackoverflow.com/questions/616645/how-do-i-duplicate-sys-stdout-to-a-log-file-in-python.

It works the first couple of times my program runs, but the 3rd time I get:

  File "C:\Users\Dave\data\Code\Python\lib\nerdlib.py", line 351, in write
    with open(self.name, "a",  encoding="utf-8") as f:

TypeError: expected str, bytes or os.PathLike object, not NoneType

It seems that even after I do sys.stdout = self.old_stdout (restoring the execution vector to the place it started from), somehow my write method is still getting called (in which case the error is expected).

But why is my method still getting called?

This error occurs in the Spyder development environment - not sure if that's related.

Code is below - it's simple enough.

# based on Triptych's answer at https://stackoverflow.com/questions/616645/how-do-i-duplicate-sys-stdout-to-a-log-file-in-python
class tee():
    ''' Tees stdout to both a log file and the terminal.

        if append is True (default), appends to existing file

        if stderr is True (default), tees stederr to same file

        Usage:
            t = tee("filename.txt")
            ...
            print(1, 2, 3)
            ...
            t.__del__() # returns sys.stdout to original state
    '''

    def __init__(self, filepath, append=True, stderr=True):

        self.old_stdout = sys.stdout
        self.old_stderr = sys.stderr
        self.name = filepath

        if (not append) or (not os.path.exists(self.name)):
            makeEmptyFile(self.name)

        sys.stdout = self

        if stderr:
            sys.stderr = self

    def write(self, text):

        self.old_stdout.write(text)

        # open file, write, then close it again (slow, but safe - permits other processes to write to same file)
        with open(self.name, "a",  encoding="utf-8") as f:
           f.write(text)

    def flush(self):
        pass

    def stop(self):
        sys.stdout = self.old_stdout
        sys.stdout = self.old_stderr
        self.name = None

    def __del__(self):
        self.stop()

@CharlesDuffy But `self.name = None` only happens AFTER the `sys.stdout = self.old_stdout`, which restores the stdout to run the old code. My write method should't be getting called at all after that. (Unless I'm terribly confused.) — nerdfever.com, Mar 13 '20 at 21:55
Ahh, gotcha. Thing is, `sys.stdout = self.old_stdout` only works for code that's going through the `sys` module every time it wants to do a write. — Charles Duffy, Mar 13 '20 at 21:56
Let's say you've got a thread started with an initializer that's something like `class MyThread(threading.Thread): def __init__(self, log_dest=sys.stdout, ...)` -- that `log_dest` won't magically change back for any already-started copy of the thread. That's just an example, intended to provide a demonstration that it's not safe to assume that there are no remaining handles after you change the sys module's reference back. — Charles Duffy, Mar 13 '20 at 21:57
@CharlesDuffy Ya, that's probably it. Makes sense. I'll test a mod with that in mind - if it works you should make that an answer. — nerdfever.com, Mar 13 '20 at 22:08

score 1 · Accepted Answer · answered Mar 13 '20 at 22:14

The danger here is that setting sys.stdout = sys.old_stdout stops code from getting a new handle on the local replacement, but it doesn't stop any old handle from working.

As a contrived example:

class MyThread(threading.Thread):
    def __init__(self, log_dest=sys.stdout):
        self.log_dest = log_dest
    def run(self):
        # ...do something-or-other, and occasionally...
        self.log_dest.write('Hey, something happened')

Because log_dest gets assigned a value when MyThread was initialized, changing sys.stdout back to its old value doesn't prevent it from still trying to use your now-invalidated object.

Yes, that was it. Changing the write method to do simply `print(text)` if `self.name is None` seems to fix it. Tricky thing to get this to work reliably. — nerdfever.com, Mar 13 '20 at 22:17

Why does my revectored sys.stdout.write() method get called after I've unlinked it?

1 Answers1