Our UI server needs to make certain downloadable files available to users. The files live on a specialized storage server. The UI server enforces some permissions and uses HTTP Basic Auth user/pwd to get the files off the storage server, but the user should never know these storage credentials so the files are streamed through our UI server to users.
We need to stream these through our UI server running Rails 4.2.11 and Ruby 2.4.9. Up until now we've been assigning an enumerator to the response body like Ruby on Rails 3: Streaming data through Rails to client , but it has been quite unreliable and we've had many cut off or incomplete downloads.
It appeared that the rack-hijack method ( https://blog.chumakoff.com/en/posts/rails_sse_rack_hijacking_api )method was newer and we hoped more reliable, but it also cuts off downloads when slower clients connect to download and it is a large file.
It appears that IO buffers are filling up and creating EOF or other errors since the download is streamed faster from the storage server than it is usually streamed through the UI by users.
I'm getting the rack-hijack IO stream (
PhusionPassenger::Utils::UnseekableSocket) and writing the results of the chunks from the GET response to it.I've tried many different http client gems and they all have similar problems, I think because it is filling the output buffer when slow clients download, not because they have problems, though the errors look that way.
When http clients have problems in this scenario they variously complain of either EOF or
error reading from socket: Could not parse data entirely (1 != 0).I've tried different versions of Passenger and tried various general solutions.
I've looked at a similar question at Rails: How to send file from S3 to remote server . In the answer, he suggests using
<obj>.stat.sizeto check on the buffer and not have it overflow and have it match the user's download speed more. However, the stream I'm given byrack-hijackdoesn't seem to have any way to check the size or buffer that I can find (PhusionPassenger::Utils::UnseekableSocket). If it exposed a way to do this, I could likely do this.I've looked through the Passenger documentation for buffering information and changed some buffering settings but it didn't help.
I've looked into the methods on the stream I get from Passenger and also on the
.to_iosettings but I'm having a hard time finding tools I think I need and having trouble finding much documentation.If I match downloads speeds from both ends then no problems. Stuff like
wget --limit-rate=2m http://localhost:3000/download_test/stream1allows me to throttle the speed of the client and I usually get an error around 180MB into the download.It's running on Amazon Linux but runs either much better or without many problems on a local Mac development machine. It's hard to tell exactly if it's still problematic or just not as much since the problem can be a little dependent on size and other factors.
# a simplified version of the code, using a public URL
def stream1
url = 'https://www.spacetelescope.org/static/archives/images/publicationtiff40k/heic1502a.tif'
response.headers['Content-Type'] = 'image/tiff'
response.headers['Content-Disposition'] = 'attachment; filename="funn.tif"'
response.headers["X-Accel-Buffering"] = 'no'
response.headers["Cache-Control"] = 'no-cache'
response.headers["Last-Modified"] = Time.zone.now.ctime.to_s
response.headers["rack.hijack"] = proc do |stream|
Thread.new do
begin
response = HTTP.get(url)
response.body.each do |chunk|
stream.write(chunk)
end
rescue HTTP::Error => ex
logger.error("while streaming: #{ex}")
logger.error("while streaming: #{ex.backtrace.join("\n")}")
ensure
stream.close
end
end
end
head :ok
end
The results I'm looking for are to make downloads reliable no matter what (reasonable) speed of the client downloading from the UI.
Thanks for any help you can give since I'm feeling stuck.
