python : redirect output to text file and display it in real time

Question

I have a python function that calls many services, and prints results of the services called in a log file. this is my code :

def coordinator():
   f = file(path,'a')
   sys.stdout = f
   /* do a lot of stuff */
   f.close()
   with open(path) as log:
        logs = log.read()
        return jsonify(log = logs)

The result of the function is returned to a jQuery getJSON function that displays the final log file in a web page :

$.getJSON('/../coordinator',
  {//parameters},
  function(data) {
    //display data.log
  }
);

My problem is that the user gets to see the log file only when the execution is finished. Is there a way to display logs as they are written in real time??

`coordinator` could start the operation in the background and return immediately with some sort of identifier for the log file. Another method could be written to read the file. There are lots of other options including highly deluxe middleware that uses something like `zeromq` to start operations and provide a publisher to track how operations are going. — tdelaney, Apr 20 '15 at 18:15

score 1 · Answer 1 · edited May 23 '17 at 12:14

1

Python's stdout is buffered, so it won't write until the buffer is full. To force an immediate write, use sys.stdout.flush(), or f.flush() in the case of your code.

edited May 23 '17 at 12:14

Community

1
1

answered Apr 20 '15 at 18:06

MattDMo

100,794
21
241
231

I already did that but I couldn't figure out how to fetch the file after f.flush() – Mahmoud Ksemtini Apr 20 '15 at 18:09
Python's `stdout` buffering is determined by whether its attached to a `pty`. OP replaces `stdout` with an open file and its the open file, not the original `stdout` that's buffered. That said, where is OP supposed to put the flushes? – tdelaney Apr 20 '15 at 18:10
I put the flushes in the services that are called by the coordinator function, but after that I didn't know how to fetch the file repetitively – Mahmoud Ksemtini Apr 20 '15 at 18:12

score 0 · Accepted Answer · answered May 04 '15 at 17:08

I finally found what I was looking for here :

http://blog.miguelgrinberg.com/post/easy-websockets-with-flask-and-gevent

So basically I created a WebSocket between the Back-end (Python/Flask) and the Front-end (HTML, jquery) to ensure continuous communication.

And with Python I created a Thread that runs asynchronously with the Master thread, fetches repetitively the content of the log file and sends them to Jquery via ajax

python : redirect output to text file and display it in real time

2 Answers2