How to store output printed on terminal console to a text file in Python

Question

I have a large python script. It prints a bunch of output on my terminal console. The problem is the print is not happening altogether. Some print statements print one blob of statements together, then under that some other part of code prints some stuff. It goes on as long as the main loop runs.

Issue is I get the output as I want but all is getting printed on console as that is where we are running the python main script.

It would be very helpful if along with the print happening at console, I can get all the output in console in same format to a text file also for retention.

Again, there are bunch of print statements occurring in different parts of the whole script. So not sure how to retain the whole output of console in same format to a final text file.

Currently running on mac terminal. But in the end the script wud run on linux shell on a server. Script is written in python — Baktaawar, Jul 15 '20 at 07:39
Have you ever thought about [logging](https://docs.python.org/3/howto/logging.html)? — AnsFourtyTwo, Jul 15 '20 at 07:40
I need to check logging. I wanted to know if doing loggin wud store the print statements output to a txt file? and in same format as it shows on console? — Baktaawar, Jul 15 '20 at 07:41
Yes, that would be possible. `print` is good in the beginning, e.g. for rapid prototyping. But as soon as your project gets larger, logging is the way to go. — AnsFourtyTwo, Jul 15 '20 at 07:43
no but in my case I am using print because its used to show the output in the end — Baktaawar, Jul 15 '20 at 07:44
@AnsFourtyTwo I looked at logging. Its mainly useful to log events like error, warning etc. It says for showing output use print. If I use logging, it wud print phrases like :Warning: , INFO based on which logging method I am caling. I don't want that. I want a clean output — Baktaawar, Jul 15 '20 at 07:49
@Baktaawar, ok, just wanted to be sure. My recommendation goes to `tee` then. — AnsFourtyTwo, Jul 15 '20 at 07:52

Vignesh · Answer 1 · 2020-07-15T09:25:28.693

3

If you want to do the redirection within the Python script, setting sys.stdout to a file object does the trick:

import sys
sys.stdout = open('file', 'w')
print('test')

A far more common method is to use shell redirection when executing (same on Windows and Linux):

$ python foo.py > file

Check this thread Redirect stdout to a file in Python?

Custom Print function for both console and file, replace all print with printing in the code.

outputFile = open('outputfile.log', 'w')

def printing(text):
    print(text)
    if outputFile:
        outputFile.write(str(text))

edited Jul 15 '20 at 09:25

answered Jul 15 '20 at 07:40

Vignesh

1,553
1
10
25

will that also print the output on console? I want the output on console and also the same output copied to a file w/o having to do redirection in shell, but just in script – Baktaawar Jul 15 '20 at 07:53
Ok I tried this. It is good, but it then doesn't print the output on console and sends it only to file. How can we have both w/o a user having to add more extra commands at console – Baktaawar Jul 15 '20 at 08:02
I did sys.stdout to file. It prints to file. How does one go back to console format. It doesnt print anything to console anymore. I tried adding sys.__stdout__. No result – Baktaawar Jul 15 '20 at 08:17
you can create a custom function for print and replace it with all prints – Vignesh Jul 15 '20 at 09:24

score 2 · Answer 2 · answered Jul 15 '20 at 07:40

2

I would rather go ahead with bash and use tee command. It redirects the output to a file too.

python -u my.py | tee my_file.txt

answered Jul 15 '20 at 07:40

bigbounty

16,526
5
37
65

how can we do this in the script. Thing is I want to take away any more addition by a user. I want them to just run the script and pass some argument. It shows the output and also stores the same in a file on a directory. The code would name the file accordingly w/o them thinking what the name of file to keep. So something which tee does but inside the script – Baktaawar Jul 15 '20 at 07:56

Roshin Raphel · Answer 3 · 2020-07-15T08:57:02.873

2

If your python script is file.py, Then use :

python3 file.py > output.txt

Or

python file.py > output.txt

Depending on your python version. This statement (>) will all the outputs of the program into the stdout to the file, output.txt

EDIT :

python3 file.py > output.txt;cat output.txt

The above line can be used to print the file output.txt after the program execution.

EDIT2 : Another possible option to use a custom print function :

f = open('output.txt')
def custom_print(e = '\n',*s)
    for i in s[:-1]:
        print(i,end=' ')
    print(s[-1],end = e)
    f.write(s)
#Your code
#
f.close()

edited Jul 15 '20 at 08:57

answered Jul 15 '20 at 07:41

Roshin Raphel

2,612
4
22
40

1

it might only send it to txt. How to also see the same output on console – Baktaawar Jul 15 '20 at 07:43
`python file.py > output.txt;cat output.txt` Will also print the file to console, if you want it to print it along with the interpreter running, you might need to use a print to file and a print to console function. – Roshin Raphel Jul 15 '20 at 07:47
Edited, check the answer again. – Roshin Raphel Jul 15 '20 at 07:49
1

ah get it. Is there a way to do the same in script and just run the script in terminal? Issue is there are a bunch of command line argument that wud be passed while running the script. I don't want to complicate it by having extra add on in the end as above – Baktaawar Jul 15 '20 at 07:51
what is s? Do I hv to write this custom_print every time where i hv a print? – Baktaawar Jul 15 '20 at 08:04
s is the argument to be print. I am afraid that is the only way, otherwise you have to try the first method – Roshin Raphel Jul 15 '20 at 08:06
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/217884/discussion-between-roshin-raphel-and-baktaawar). – Roshin Raphel Jul 15 '20 at 08:07
after doing sys.stdout and file.. now If I want to revert back to terminal how do I do it? I now only sends to a file and no console output – Baktaawar Jul 15 '20 at 08:07
sys.stdout = original_stdout use this to get output back to console, read here https://stackabuse.com/writing-to-a-file-with-pythons-print-function/ – Vignesh Jul 15 '20 at 09:37

score 1 · Answer 4 · answered Jul 15 '20 at 07:38

1

you have to add file argument to the print() function

print('whatever', file = file_name)

answered Jul 15 '20 at 07:38

Aleksander Ikleiw

2,549
1
8
26

wud I have to add this to all print statements? I have like a tonne of such prints. I also want the output to be printed in console too. – Baktaawar Jul 15 '20 at 07:39
@Baktaawar yes you would have to add to all. If you are using the `Pycharm` it is very easy. It will be printed and saved to a file – Aleksander Ikleiw Jul 15 '20 at 07:40
@Baktaawar there is also one way `sys.stdout = open('file_name'', 'a')` – Aleksander Ikleiw Jul 15 '20 at 07:42
using VScode.. currently testing the code on jupyter. But eventually code has to be run as a script in console. Quick check- will this adding of file to all print also print the output to console as usual – Baktaawar Jul 15 '20 at 07:42
is this sys.stdout only needed once at the top? Or after each print statement – Baktaawar Jul 15 '20 at 07:43
@Baktaawar at the top, do not forget to import `sys` – Aleksander Ikleiw Jul 15 '20 at 07:43
checked stdout thing. It doesnt print it on console anymore. And all output goes to file. I want both console and file. But don't want a user to add extra unix command like tee or cat to do that. The script should do automatically while printing on console and also sending it to file – Baktaawar Jul 15 '20 at 08:03
I did sys.stdout to file. It prints to file. How does one go back to console format. It doesnt print anything to console anymore. I tried adding sys.__stdout__. No result – Baktaawar Jul 15 '20 at 08:17

How to store output printed on terminal console to a text file in Python

4 Answers4