I am running an incredibly time consuming python script (~20 hours). I like it to give me periodic updates about how it's coming along. The print statements are so infrequent that they could take 5 minutes each and it would hardly effect run time (~10 in all that time). I want a record of the output so I pipe it to a file. I use tmux and in a tmux session run:
python myscript.py > output.txt
Then I detach from the session and let it do its thing. The issue is there is a tremendous lag between when the script should print and when I see the output in the file. I tried to use tee because that would be perfect for this setup but I ran into the same issue. When I ran:
python myscript.py | tee output.txt
Nothing was printed to stdout or appended to the file. I think this might be because of how piping works. Is there some way I can see the output after each newline symbol rather than waiting? Can I do this with tee? In general how does piping decide when to send output?
