How do I pass through or wrap the print command(stdout) so that print also calls a function every call?

Question

I am trying to automating a long running job, and I want to be able to upload all console outputs to another log like on CloudWatch Logs. For the most part this can be done by making and using a custom function instead of print. But there are functions in MachineLearning like Model.summary() or progress bars while training that outputs to stdout on their own.

I can get all get all console outputs at the very end, via an internal console log. But what I need is real-time uploading of stdout as its called by whomever. So that one can check the progress by taking a look at the logs on Cloudwatch instead of having to log into the machine and check the internal console logs.

Basically what I need is:

From: call_to_stdout -> Console(and probably other stuff)
To:   call_to_stdout -> uploadLog() -> Console(and probably other stuff)

pseudocode of what I need

class stdout_PassThru:
   def __init__(self, in_old_stdout):
      self.old_stdout = in_old_stdout

   def write(self, msg):
      self.old_stdout.write(msg)
      uploadLogToCloudwatch(msg)

def uploadLogToCloudwatch(msg):
   # Botocore stuff to upload to Cloudwatch

myPassThru = stdout_PassThru(sys.stdout)
sys.stdout = myPassThru

I've tried googling this, but the best I ever get is stringIO stuff, where I can capture stdout, but I cannot do anything with it until the function I called ends and I can insert code again. I would like to run my upload Log code everytime stdout is used.

Is this even possible? Please and thank you.

EDIT: Someone suggested redirect/output to file. The problem is that, that just streams/writes to the file as things are outputted. I need to call a function that does work on each call to stdout, which is not a stream. If stdout outputs everytime it flushes itself, then having the function call then would be good too.

Does this answer your question? [Redirect stdout to a file in Python?](https://stackoverflow.com/questions/4675728/redirect-stdout-to-a-file-in-python) — joshmeranda, Sep 07 '20 at 12:12

score 0 · Answer 1 · answered Sep 08 '20 at 04:19

I solved my problem. Sort of hidden in some other answers. The initial problem I had with this solution is that when it is tested within a Jupyter Notebook, the sys.stdout = myClass(sys.stdout) causes Jupyter to... wait? Not sure but it never finishes processing the paragraph.

But when I put it into a python file and ran with python test.py it ran perfectly and as expected.

This allows me to in a sense pass thru calls to print, while executing my own function every call to print.

def addLog(message):
    # my boto function to upload Cloudwatch logs

class sendToLog:
    def __init__(self, stream):
        self.stream = stream
    def write(self, o):
        self.stream.write(o)
        addLog(o)
        self.stream.flush()
    def writelines(self, o):
        self.stream.writelines(o)
        addLog(o)
        self.stream.flush()
    def __getattr__(self, attr):
        return getattr(self.stream, attr)

sys.stdout = sendToLog(sys.stdout)

How do I pass through or wrap the print command(stdout) so that print also calls a function every call?

1 Answers1