Django - Send Output Continually

Question

I want to start processing some files from a django view and I want to be able to send the name of the files to the browser as they are processed. Is there a way to do this (easily)? I could probably do this using threads and ajax calls, but I want the simplest solution for now.

Answer 1

I found what I needed in an answer from one of the links that Andre Miller provided.

I found out that's possible to pass an iterator to HttpResponse so I used this code and it worked:

def import_iter():
    """ Used to return output as it is generated """
    # First return the template
    t = loader.get_template('main/qimport.htm')
    c = Context()
    yield t.render(c)
    # Now process the files
    if req.method == 'POST':
        location = req.POST['location']
        if location:
            for finfo in import_location(location):
                yield finfo+"<br/>"

return HttpResponse(import_iter())

Answer 2

You would need to use some sort of queuing process if you want to kick off the task when the view is rendered, otherwise the process will finish first before anything is returned to the browser.

Once the task is running asynchronously you could use either AJAX to update the page with the latest status or simply use a meta-refresh inside the page to load the new content.

There is Django queue server here you could use:

http://code.google.com/p/django-queue-service/

It would seem that this question has also been asked a few times before:

• How to best launch an asynchronous job request in Django view?
• Is there any way to make an asynchronous function call from Python [Django]?
• How do you do something after you render the view? (Django)

Answer 3

We are in 201X

Yes, you should use WebSockets or Ajax calls !!

Since you were asking(for the record purpose) for some streaming solution in Django you can use StreamingHttpResponse which Django supports out of the box.

https://docs.djangoproject.com/en/dev/ref/request-response/#django.http.StreamingHttpResponse

The StreamingHttpResponse class is used to stream a response from Django to the browser. You might want to do this if generating the response takes too long or uses too much memory. For instance, it’s useful for generating large CSV files.

Answer 4

If you clear the output buffer, then you should be able to see what has been processed.

Answer 5

First of all, make sure you output a Connection: Keep-Alive header, after which you just have to make sure that the script output isn't being buffered. In Python, you can use the cgi module's cgiprint function to ensure that Python's buffer is cleared, but you should also check the web server configuration, as some will buffer all output until the script finishes running.