50

Okay. I have completed my first python program.It has around 1000 lines of code. During development I placed plenty of print statements before running a command using os.system() say something like,

print "running command",cmd
os.system(cmd)

Now I have completed the program. I thought about commenting them but redirecting all these unnecessary print (i can't remove all print statements - since some provide useful info for user) into a log file will be more useful? Any tricks or tips.

vvvvv
  • 25,404
  • 19
  • 49
  • 81
webminal.org
  • 44,948
  • 37
  • 94
  • 125
  • 1
    i think simple code snippet provided by Michel will fulfill my requirements ...thank you all – webminal.org Mar 25 '10 at 08:02
  • I tried the accepted answer's code but didn't really work. https://stackoverflow.com/questions/54049235/redirect-both-print-and-os-system-to-log-file-while-maintaining-the-origin –  Jan 05 '19 at 05:34
  • related: https://stackoverflow.com/q/14058453 – djvg Jun 30 '22 at 06:52

10 Answers10

72

Python lets you capture and assign sys.stdout - as mentioned - to do this:

import sys
old_stdout = sys.stdout

log_file = open("message.log","w")

sys.stdout = log_file

print "this will be written to message.log"

sys.stdout = old_stdout

log_file.close()
Nikunj Rathi
  • 3
  • 3
Michael
  • 3,498
  • 5
  • 27
  • 32
  • 13
    You can also do -- sys.stdout = sys.__stdout__ -- instead of using old_stdout. –  Mar 25 '10 at 06:56
  • This is a messy hack, but certainly possible. If you **do** use it, reset `sys.stdout` to its original value in the `finally` block of a `try/finally`. – Mike Graham Mar 25 '10 at 14:45
  • 2
    @007brendan, that assumes that `sys.stdout` *starts* as `sys.__stdout__`. That isn't a great assumption, someone else could be redirecting `sys.stdout` for a purpose like this or for a more sane use, like a system where a *whole app's* stdout really shouldn't be `sys.__stdout__`. – Mike Graham Mar 25 '10 at 14:57
  • I tried the code but didn't really work. What's wrong with this? https://stackoverflow.com/questions/54049235/redirect-both-print-and-os-system-to-log-file-while-maintaining-the-origin –  Jan 05 '19 at 05:35
  • 1
    works! python3.6 also works, need to be careful to close the opened file at the end – user5824405 Jan 02 '20 at 14:41
  • Michael's answer works on python3.8. Can define a custom print function using this. – RAVI D PARIKH Dec 25 '20 at 20:23
65

You should take a look at python logging module

EDIT: Sample code:

import logging

if __name__ == "__main__":
    logging.basicConfig(level=logging.DEBUG, filename="logfile", filemode="a+",
                        format="%(asctime)-15s %(levelname)-8s %(message)s")
    logging.info("hello")

Produce a file named "logfile" with content:

2012-10-18 06:40:03,582 INFO     hello
  • 1
    +1 for the logging module. Gives you wayyyy more control than print statements. –  Mar 25 '10 at 06:57
  • 26
    Your solution doesn't in fact answer the question of "how to redirect output" at all! Go to that page - search (in vain) for any hint of how to redirect logging to a file. The actual solution is on the page for logging.config - but I defy you to locate it there. http://onlamp.com/pub/a/python/2005/06/02/logging.html might be a better example. – Tom Swirly Oct 17 '12 at 21:39
  • 3
    @TomSwirly at the bottom of the page: a link to http://docs.python.org/library/logging.handlers.html#module-logging.handlers gives you a page where `FileHandler` is explained –  Oct 18 '12 at 04:35
  • 2
    @TomSwirly and the page also explain `basicConfig` –  Oct 18 '12 at 04:44
  • 7
    You're right, it is at the bottom of that page, but by being wrong I improved your answer, so it worked out! – Tom Swirly Oct 18 '12 at 06:40
12

  • Next time, you'll be happier if instead of using print statements at all you use the logging module from the start. It provides the control you want and you can have it write to stdout while that's still where you want it.

  • Many people here have suggested redirecting stdout. This is an ugly solution. It mutates a global and—what's worse—it mutates it for this one module's use. I would sooner make a regex that changes all print foo to print >>my_file, foo and set my_file to either stdout or an actual file of my choosing.

    • If you have any other parts of the application that actually depend on writing to stdout (or ever will in the future but you don't know it yet), this breaks them. Even if you don't, it makes reading your module look like it does one thing when it actually does another if you missed one little line up top.
    • Chevron print is pretty ugly, but not nearly as ugly as temporarily changing sys.stdout for the process.
    • Very technically speaking, a regex replacement isn't capable of doing this right (for example, it could make false positives if you are inside of a multiline string literal). However, it's apt to work, just keep an eye on it.

  • os.system is virtually always inferior to using the subprocess module. The latter needn't invoke the shell, doesn't pass signals in a way that usually is unwanted, and can be used in a non-blocking manner.

Mike Graham
  • 73,987
  • 14
  • 101
  • 130
6

You can create a log file and prepare it for writing. Then create a function:

def write_log(*args):
    line = ' '.join([str(a) for a in args])
    log_file.write(line+'\n')
    print(line)

and then replace your print() function name with write_log()

AisZuk
  • 61
  • 1
  • 2
2

You can redirect replace sys.stdout with any object which has same interface as sys.stdout, in that object's write you can print to terminal and to file too. e.g. see this recipe http://code.activestate.com/recipes/119404-print-hook/

Anurag Uniyal
  • 85,954
  • 40
  • 175
  • 219
  • Though applicable, that recipe is stylistically, semantically, and conceptually awful. – Mike Graham Mar 25 '10 at 14:50
  • @Mike Graham, hey i wrote that 8 years ago :), anyway why is that so awful ? – Anurag Uniyal Mar 26 '10 at 00:45
  • I would avoid using something like that if I could possibly do so for a couple reasons. It changes a global value (`sys.stdout`) for local use and may have accidental, far-off, unexpected effects (this is the reason people avoid using global state). When it's done, it restores the value to `sys.__stdout__`, which may and may not be what `sys.stdout` started as depending on if someone else was doing this sort of trick. It does not have a usage such that an unexpected exception won't leave `sys.stdout` stuck at an unwanted value later. – Mike Graham Mar 26 '10 at 01:14
  • That being said, I'm sorry to have been uncivil. Though I honestly believe the best solution will not use this kind of technique, I was unduly harsh in my criticism. – Mike Graham Mar 26 '10 at 01:15
  • @Mike Graham that is fine, criticism is good, and that hook is hack to divert all the prints of app if need be, in practical situation I have used it on some old apps which have print littered all around – Anurag Uniyal Mar 26 '10 at 02:50
2

A simple way to redirect stdout and stderr using the logging module is here: How do I duplicate sys.stdout to a log file in python?

Community
  • 1
  • 1
blokeley
  • 6,726
  • 9
  • 53
  • 75
  • 1
    That solution is disgusting and not even entirely applicable (in that it writes to stdout *and* a file; OP only wants to write to a file.) – Mike Graham Mar 25 '10 at 15:02
  • 3
    Please could you explain why this is "disgusting"? I know redirecting stdout is a bad idea but sometimes it's worth doing: this week I did it to run django on a server overnight for testing, and django's runserver command prints to stdout. Also, please test your opinion before posting because the solution in question does _not_ write to stdout. I have just tested it again. – blokeley Mar 25 '10 at 16:40
  • This is disgusting because it uses globals. This means that when you read `print foo` it doesn't do what you expect it to without reading where the global was changed, which might not even be in the same file as the print statement. It's unmodular because multiple things can't all do this sanely. You even acknowledge that it's a bad idea. When something should be printed to stdout but you want to redirect it, there are tools for this (like `>` on the shell); when something should be written somewhere other than stdout, you should make your code write there instead. – Mike Graham Mar 25 '10 at 19:18
  • 1
    I apologize about my statement about the code writing to a file and stdout. I was looking at a different solution in the same thread as you linked. My mistake. – Mike Graham Mar 25 '10 at 19:19
  • Apology accepted. Thanks for the clarification. I agree one should avoid messing with globals. I have updated my original post to point out that the logging module should be used directly wherever possible. This is what I always intended but didn't say it explicitly until now. – blokeley Mar 26 '10 at 08:00
2

there are many way to write output into the '.log' file

Logging is a means of tracking events it happen when some file runs. Is also indicate that certain events have occurred.

import logging
logging.basicConfig(filename='example.log', encoding='utf-8', level=logging.DEBUG)
logging.debug('This is debug message')
logging.info('This is information message')
logging.warning('This is warning message')
logging.error('This is warning message')

another method to use to reduce all that thing sinple what ever you print to the console that all will be saved to the ''log'' file

python abc.py > abc.log

by using this method you can write everything to the log file

Sanjay Sheel
  • 94
  • 7
1

Putting your own file-like in sys.stdout will let you capture the text output via print.

Ignacio Vazquez-Abrams
  • 776,304
  • 153
  • 1,341
  • 1,358
0

Just a note about append vs write mode. Change filemode to "w" if you would like it to replace log file. I also had to comment out the stream. Then using logging.info() was outputting to file specified. 

if __name__ == '__main__':
    LOG_FORMAT = '%(asctime)s:%(levelname)s ==> %(message)s'
    logging.basicConfig(
        level=logging.INFO,
        filename="logfile",
        filemode="w",
        format=LOG_FORMAT
        #stream=sys.stdout
    )
Christopher Adams
  • 1,240
  • 10
  • 14
0
def log(txt):
    f = open(__file__ + '.log', "a")
    f.write(txt + '\r\n')
    f.close()

Usage:

log('Hello World!')

Example:

python3 helloworld.py

Will append to file ./helloworld.py.log. If file doesn't exist, it will create it.

Jeff Luyet
  • 424
  • 3
  • 10