How to flush current output in RApache?

Question

I'm testing using RApache as an SSE (Server Sent Events) and similar (long poll, comet, etc.) back-end. I seem to be stuck on how to flush my output. Is it possible?

Here is my test R script:

setContentType("text/plain")

repeat{
    cat(format(Sys.time()),"\n")
    #sendBin(paste(format(Sys.time()),"\n"))
    flush(stdout())
    Sys.sleep(1)
    }

My Rapache.conf entry is:

<Location /rtest/sse>
    Options -MultiViews
    SetHandler r-handler
    RFileHandler /var/www/local/rtest/sse.r
</Location>

And I test it using either wget or curl:

wget -O - http://localhost/rtest/sse
curl http://localhost/rtest/sse

Both just sit there, meaning nothing is being sent.

Using sendBin() made no change, and neither did using flush().

If I change repeat to for(i in 1:5) then it sits there for 5 seconds and then shows 5 timestamps (spaced one second apart). So, I believe everything else is working fine and this is purely a buffering issue.

UPDATE: Looking at this with fresh eyes after 5 months, I think I could have described the problem more clearly: the problem is that RApache appears to be buffering all the output, and not sending anything until the R script exits. To be useful for streaming it has to send data out of Apache and on to the client each time flush() is called, i.e. while the R script is still running. So, my question is: is there a way to get RApache to behave like that?

UPDATE 2 I tried adding flush.console() before or after the flush(stdout()) but no difference. I also tried setStatus(status=200L) at the top. And I tried SERVER$no_cache=T;SERVER$no_local_copy=T; at the top of the script. Again it made no difference. (Yes, none of those should have helped, but it never hurts to try!)

Here is a link to how PHP implements flush when it is running as an Apache module: http://git.php.net/?p=php-src.git;a=blob;f=sapi/apache2handler/sapi_apache2.c#l290 I think the key point is that there is a call to ap_rflush(r). I'm guessing that RApache is not making the ap_rflush() call.

Answer 1

You are passing the wrong MIME type. Try changing with

setContentType("text/event-stream")

EDIT1:

this is the attempt, (still unsuccessful) I mentioned in the comment below, to implement SSE in Rook.

<%
  res$header('Content-Type', 'text/event-stream')
  res$header('Cache-Control', 'no-cache')
  res$header('Connection', 'keep-alive')
  A <- 1

  sendMessage <- function(){
    while(A<=4){
      cat("id: ", Sys.time(), "\n", "data: hello\n\n", sep="")
      A <- A+1
      flush(stdout())
      Sys.sleep(1)
    }
  }

-%>

<% sendMessage() %>

the while loop condition was supposed to be always TRUE but I'm having your same problem so I had to do a finite loop...

The good new is I DO have data reaching the browser. I can tell by looking, in developer tools, at the Content-Length in the Response Header section. it says 114 for the above code and you change, say, "Hello" in "Hello!" it'll say 118.

The js code is: (you'll need JQuery as well)

$(document).ready(function(){

  $("button").click(function(){

    var source = new EventSource("../R/sse.Rhtml");

    source.onopen = function(event){
      console.log("readyState: " + source.readyState);
    }

    source.onmessage = function(event){
      $("#div").append(event.data);
    };

    source.onerror = function(event){
      console.log(event);
    };

  });


});

So, in essence

1) The connection is open (readyState 1)

2) Buffering is still there

3) Data (after buffering) reaches the browser but an error happens in receiving them properly.

EIDT2:

it's interesting to note that brew()ing the above .Rhtml file the output is not buffered. There must be a configuration the in the web server (both the R internal and Apache) that buffer the data flows.

As a side note, flush is not even needed, cat's output defaults to stout(). So the options are:

1. Web server configuration
2. The R equivalent of the PHP ob_flush(); which is always used in any PHP implementation I've seen. this is example