3

I recently discovered rails streaming, and having read the various documentation, I tried to add it to one of my controllers like so:

def index
    render :stream => true
end

My template file currently looks like this:

<%10.times do%>
  <p>
    I should get streamed ERB...
    <%sleep 0.5%>
  </p>
<%end%>

But instead of displaying a message every 0.5 seconds, it waits 5 seconds and displays the whole page! I have checked this behavior in a browser and using curl.

I am using unicorn on OSX, and yes, I configured unicorn.rb for streaming:

listen 3000, :tcp_nodelay => true, :tcp_nopush => false
worker_processes 1

If you want to see my layout file, it looks like this:

<!DOCTYPE html>
<html>
<head>
  <title>StreamingTest</title>
  <%= stylesheet_link_tag    "application" %>
  <%= javascript_include_tag "application" %>
  <%= csrf_meta_tags %>
</head>
<body>

<%= yield %>

</body>
</html>

I tried disabling all gems (except rails and postgres), and I pared my controller, layout, and template down to the bare minimum, without any success.

I even went so far as to download a complete streaming demo I found (https://github.com/slim-template/slim-streamingtest), and when I ran that, it didn't stream either! (Note to anybody trying to run that demo: I had to change the gemfile source to use https instead of http before I could "bundle install", and I had to "bundle update sprockets" before it would run)

I note that somebody else (using Puma) appears to have a similar problem, which has not been successfully resolved:

Rails Streaming not Streaming

It may be that whatever works for me will also work for them.

Streaming would really help our app, if I could just get it to work, but I can't even get demo streaming apps to run correctly! I suspect it might have something to do with my dev environment, but the problem remains when I deploy my app to Heroku, and I'm out of ideas. Any help would be greatly appreciated . . .

Thanks!

Community
  • 1
  • 1
dacoinminster
  • 3,637
  • 2
  • 18
  • 23

2 Answers2

2

I tried the demo repo you mentioned and the behaviour is as expected. In the example you show it will stop streaming the template, then wait for it to be fully rendered after 5 sec and send the rest. If you use curl:

# curl -i http://localhost:3000

HTTP/1.1 200 OK
Date: Thu, 21 Apr 2016 17:29:00 GMT
Connection: close
Cache-Control: no-cache
Transfer-Encoding: chunked
Content-Type: text/html; charset=utf-8
X-UA-Compatible: IE=Edge
X-Runtime: 0.018132

<!DOCTYPE html>
<html>
  <head>
    <title>StreamingTest</title>
    <link href="/stylesheets/application.css" media="screen" rel="stylesheet" type="text/css" />
    <script src="/javascripts/application.js" type="text/javascript"></script>
    <meta content="authenticity_token" name="csrf-param" />

Then it waits for 5 seconds for the template to finish its rendering and then sends it.

  ...
  <p>
    I should get streamed...
  </p>
</body></html>

That's what's supposed to happen. It will not send a part of the partial every 0,5 seconds.

The main goal of streaming with templates is for the browser to receive the headers before the rest of the content so that it makes it possible to save some time in loading assets. As stated in the docs.

Streaming inverts the rendering flow by rendering the layout first and streaming each part of the layout as they are processed. This allows the header of the HTML (which is usually in the layout) to be streamed back to client very quickly, allowing JavaScripts and stylesheets to be loaded earlier than usual.

It's not going to send the partial unless it's been fully rendered.

However, to show you how to get the kind of behaviour you're expecting you could change the body of application.html.slim layout like this:

  body
    = yield
    = render 'application/other'

And create a partial named _other.html.erb with the following content

<% sleep 1 %>
<p>Me too!</p>

And now you'll see in curl that it renders the beginning of the layout, waits for the rendering to be finished on index.html.erb, sends it and shows up in curl and then waits for the _other.html.erb partial to be finished rendering to end the request body. Voilà.

You could also directly feed the request body, to send content every few milliseconds, you can do the following:

def index
  headers['Cache-Control'] = 'no-cache'
  self.response_body = Enumerator.new do |yielder|
    10.times do
      yielder << "I should be streamed...\n"
      sleep 0.3
    end
  end
end

This is however not meant to send html, although you could by using render_to_string but then you'll have to modify quite a lot in your code.

Note that all this is Rails 3.x, for 4.x you'll have to use ActionController::Live. In the case of ActionController::Live, there are examples on how to listen to the server sent events in js to render client side such as this mini-chat.

Marc Lainez
  • 3,070
  • 11
  • 16
  • Thanks so much for answering @marc-lainez! I was beginning to think I would never get an answer, even with the bounty I attached! – dacoinminster Apr 21 '16 at 22:21
  • I had hoped that I could use "render :stream => true" to keep my page from timing out (I'm using Heroku, which won't let me increase the timeout, and the page was taking a REALLY long time to build), but the page still timed out, even with streaming. It sounds like that is expected behavior? I worked around this problem by optimizing the page to load faster, so I don't need an answer as badly as when I started, but I'm awarding you the bounty since you actually took the time to help! – dacoinminster Apr 21 '16 at 22:25
  • I understand the kind of situation you'd end up with and why you'd think it's a good idea to use stream or server sent events to solve it but personally I don't think it's the right approach. What you're describing seems to be performance problems when fetching data and rendering large partials. The first step would be to optimize database queries to the extreme, then maybe use caching. Another approach would be to fetch some of that content with AJAX requests instead of a regular template rendering. If you post another question on those topics I'd be happy to help. Best. – Marc Lainez Apr 22 '16 at 03:58
  • Yup - the problem was showing a large table of users (25 at a time), but multiple calls to an external API were used to display each user, which took multiple seconds each. I was able to remove the API calls (and some of the displayed data) and display that data instead only when someone clicks on the user. That got rid of the performance problem entirely, at the expense of a slightly less useful page. Thanks again for your help! – dacoinminster Apr 22 '16 at 15:57
0

There's a Rails bug opened by one of the Puma maintainers: #23828 "ActionController::Streaming doesn't stream". The bug has been open since 2016 and remains open in March of 2023. From my reading the feature is not currently usable. The accepted answer here uses ActionController::Live and is likely still viable.

Based on commentary there it may get fixed after rails switches to rack 3. That process appears to be ongoing now.

Posting this for anyone who tries to use "stream: true" in the intervening timeframe.

tgf
  • 2,977
  • 2
  • 18
  • 29