0

In my application, the user can launch a crawl action from the interface who will be executed in a thread on the server-side. This crawl is performed in a while loop and will send some information (for the moment, as a print in the console).

What I want to do is to send this information displayed in the console to the view. This diagram sum up the process 

View                                     Server
  |                                         |
  |    Call the async function with ajax    |   crawl   Thread
  |---------------------------------------->|------------>|
  |                                         |             |
  |                                         |             |
  |          send back information          | information |
  |<----------------------------------------|<------------|
  |                                         |            ...
  |------|                                  |
  |      | refresh view                    ...
  |      |
  |<------   
 ...

Is there a way to do this ? I have no idea how to do this (signals ? websockets ?)
Thanks !

Kobz
  • 469
  • 6
  • 17
  • Read about it here please http://stackoverflow.com/questions/6599783/django-frontend-to-run-subprocess-output-to-browser-window – Eugene Soldatov Apr 18 '14 at 10:24
  • How "enterprisey" are you looking for? There are quicker hacks, but if you're looking for a big scalable solution, there are ways to build what you want using celery/rabbitmq to handle the backend and then possibly nodejs or gevent (or another capability) to handle the websockets (which would allow you to make an async call and then the push from the server) – Foon Apr 18 '14 at 10:46
  • Eugene : My main problem here isn't to display the result from the terminal, it's to send asynchronous response from the server to the view without reloading the page. Foon : Well, it depends on the crawling delay which have been parameterized, it could be a message sent from the server to the view each hours like it could be each 10 seconds. I already heard about celery but never used it. From what I red about nodejs, it's Javascript server oriented so, if I'm not mistaking, I'll be able to send asynchronous response from the server to the view right ? – Kobz Apr 18 '14 at 11:43

1 Answers1

0

There are basically two approaches you can take: polling or pushing.

If you're going to poll, which is the easier choice, you can use several things, from updating the DOM with jQuery to something more elegant like Observables/ObservableArrays in Knockout.js.

If you're going to push, I would recommend using socket.io and Redis in a pub/sub fashion.

Either way, you're going to end up using JavaScript to update the DOM.

Brandon Taylor
  • 33,823
  • 15
  • 104
  • 144
  • What I'm trying to do is pushing and not polling, basically, it's a 'terminal-like' displayed in the web-interface which is displaying the result of an asynchronous operation performed by the server (results of a scheduled crawl). I managed to set up Celery in order to crawl. I expect a kind of real-time results, that's the reason why I can't do polling, because there will be an interval (between 2 requests from the client) where I'll have to wait for the next request in order to have new informations. – Kobz Apr 18 '14 at 15:07
  • Gotcha. If you store the results of those operations in a cache, like Redis, etc, you can push that information down to the client on a schedule or as soon as it becomes available. – Brandon Taylor Apr 18 '14 at 17:07
  • After some researches, I chose to finally do a simple polling. Why ? Because I'm really sad (not to say something vulgar) to see that indispensables tools such as Socket IO, redis-py, pika and so on aren't still compatibles with Python 3 (which was released in 2008). – Kobz Apr 22 '14 at 08:34
  • Don't feel bad. I still use Python 2.7 for all of my projects because I have 3rd party dependencies that aren't Python 3 compatible yet. I always want to use the latest and greatest too, but sometimes we just have to be patient. – Brandon Taylor Apr 22 '14 at 15:07