1

I am using Spyder for Python and sometime I would like to print the console into a log file (in cases where the output is quite long) and sometimes I just want to have the output at the console. For this purpose I use the following construction in my Python files:

In the beginning of the file:

import sys
# specify if the output should be printed on a separate log file or on the console
printLogToFile = False

if printLogToFile == True:
    #Specify output file for the logs
   sys.stdout = open('C:/Users/User 1/logfile.txt', 'w')

At the end of the file:

# Close the log file if output is printed on a log file and not on the console
if printLogToFile == True:
    sys.stdout.close()
    sys.stdout = sys.__stdout__

Basically, whenever my boolean variable printLogToFile has the value False then everything is printed on the console as it should and whenever it has the value True everything is printed into the logfile. However, once I run just once the file with printLogToFile=True this can't be reversed any longer. Even when the variable has the value False it still prints everything into the log file and not onto the console. What is even more strange is that also for other Python files, that do not have any connection to this file, the console is not printed any longer onto the console. The only way to solve this problem is to close Spyder and restart it again.

Do you have any idea why this is happening and how to avoid this? I'd appreciate every comment.

PeterBe
  • 700
  • 1
  • 17
  • 37
  • 1
    The first thing that springs to mind is that the console in Spyder is an [IPython](https://ipython.readthedocs.io/en/stable/index.html) console, not a plain Python one. It doesn't surprise me that the redirection affects other Python files you have open in Spyder because they all share the same console. As an initial workaround you should be able to restart the kernel inside Spyder rather than having to restart the whole IDE. – nekomatic Jul 06 '21 at 11:11
  • Next, the [docs for `sys.__stdout__`](https://docs.python.org/3/library/sys.html?highlight=__stdout__#sys.__stdout__) say *It can also be used to restore the actual files to known working file objects in case they have been overwritten with a broken object. However, the preferred way to do this is to explicitly save the previous stream before replacing it, and restore the saved object.* – nekomatic Jul 06 '21 at 11:12
  • Thanks nekomatic for your answer. Restarting the kernel solves the problem temporary. But I have to do it again and again. I do not understand your second comment. I still think it is kind of weird that the commands can't be reversed after using it altough I clearly close the stream at the end of the file. – PeterBe Jul 06 '21 at 11:54
  • The second comment means you should save the original stdout with (e.g.) `prev_stdout = sys.stdout` first, then restore it with `sys.stdout = prev_stdout` afterwards. I suspect the problem is that IPython does something with stdout, so that just setting it to `sys.__stdout__` after you're done redirecting it doesn't restore it properly. If I type `sys.stdout` in the Spyder console it returns an `ipykernel.iostream.OutStream` object, but `sys.__stdout__` is something different. – nekomatic Jul 06 '21 at 12:26
  • Congratulations for posting a question tagged `spyder` that actually is about using Spyder though :-) – nekomatic Jul 06 '21 at 14:14
  • Thanks for your comment nekomatic. I do not understand what you mean by "Congratulations for posting a question tagged spyder that actually is about using Spyder though :-)". The question is about how to print the output of the Spyder console to a log file and why this is not reversible with my currently used approach – PeterBe Jul 06 '21 at 14:45
  • 1
    Just a joke, sorry. Mostly when someone tags their question `spyder` it's not about Spyder at all, just that they happen to be using Spyder as their IDE. This question actually does relate to a difference in behaviour between running a script in Spyder and running it from the command line. – nekomatic Jul 06 '21 at 15:01

1 Answers1

1

The console in Spyder is an IPython console, not a plain Python console, so I think IPython is doing something with stdout that causes your approach to fail.

The docs for sys.__stdout__ say

It can also be used to restore the actual files to known working file objects in case they have been overwritten with a broken object. However, the preferred way to do this is to explicitly save the previous stream before replacing it, and restore the saved object.

In other words, try:

if printLogToFile:
    prev_stdout = sys.stdout
    sys.stdout = open('C:/Users/User 1/logfile.txt', 'w')

# code that generates the output goes here

if printLogToFile:
    sys.stdout.close()
    sys.stdout = prev_stdout

As an alternative, based on this answer and this answer assuming Python >= 3.7, you can use contextlib and a with statement to selectively capture the output of some of your code. This seems to work for me in Spyder 4 and 5:

from contextlib import redirect_stdout, nullcontext

if printLogToFile:
    f = open('myfile.txt', 'w')
    cm = redirect_stdout(f)
else:
    cm = nullcontext()

with cm:
    # code that generates the output goes here

If you want to execute the whole of your Python script myscript.py and capture everything it outputs, it's probably easier to leave your script unmodified and call it from a wrapper script:

# put this in the same folder as myscript.py

from contextlib import redirect_stdout

with redirect_stdout(open('myfile.txt', 'w')):
    import myscript

If you want anything more flexible than that, it's probably time to start using logging.

nekomatic
  • 5,988
  • 1
  • 20
  • 27
  • Thanks nekomatic for your answer. What do you mean by `with cm: # code whose output should be captured goes here print('2+2={}'.format(4))`? My code is quite long (more than 2000 lines). Should all of this code be placed within the `with cm` section? This would not be so good because you have an additional indentation there – PeterBe Jul 06 '21 at 11:52
  • Thanks for your answer nekomatic. I think I will just restart the kernel and use my old solution as I would like to have everything in one file (without many additional indentation). Maybe I will find a way of really closing the sys.stout because apparently this is not really working (I also tried your approach with `# Close the log file if output is printed on a log file and not on the console if printLogToFile == True: sys.stdout.close() prev_stdout = sys.stdout sys.stdout = prev_stdout sys.stdout = sys.__stdout__` but it did not change anything) – PeterBe Jul 06 '21 at 13:34
  • Don't use `sys.__stdout__` at all, just do `sys.stdout = prev_stdout` after the code whose output you want to capture. – nekomatic Jul 06 '21 at 14:12
  • Thanks nekomatic for your answer. When using the code at the end of the file `# Close the log file if output is printed on a log file and not on the console if printLogToFile == True: sys.stdout.close() prev_stdout = sys.stdout sys.stdout = prev_stdout` I get the following error "ValueError: I/O operation on closed file." – PeterBe Jul 06 '21 at 14:42
  • To avoid further confusion I've edited the answer again. Hopefully one of these approaches will work for you. – nekomatic Jul 06 '21 at 14:58
  • Thanks a lot nekomatic for your answer. Now using your approach with prev_stdout everything works as it should. I really appreciate your tremendous help. I upvoted and accepted your answer. – PeterBe Jul 06 '21 at 15:23