3

I think I want to limit the maximum file size and form payload that can reach a Rails 4 application, hosted on Heroku with the puma app server. I'd also like to kill these requests before they complete to free up server resources.

I'm assuming this can't be done in Rack, because it executes after the request has completed uploading?

With that in mind, it seems like the job of the web server, but I couldn't find mention of this in Puma. Does it have a mechanism I've missed for handling this kind of thing?

Alternatively would phusion passenger with nginx or apache handle this better?

http://httpd.apache.org/docs/1.3/mod/core.html#limitrequestbody http://www.cyberciti.biz/faq/apache-limiting-upload-size/

Why care:

  • memory exhausted by large texts being turned into large ruby strings
  • cpu wasted scanning very large strings doing sql escaping or html escaping or application validations
  • server unable to accept new connections due to large backlog of clients uploading huge files (or huge form data)

Bonus points: the next step seems to be temp-banning repeat offenders in a firewall so they can't hit the web server. What technologies are good for that on Heroku?

Edit: relevant other thread Protect yourself against Dos attacks

Community
  • 1
  • 1
nruth
  • 1,068
  • 7
  • 22

1 Answers1

2

It sounds like you're still choosing stack for this feature/app given that you're talking switching between nginx or apahce, and/or passenger or heroku.

The best way to combat this ahead of time is using client side validation of file sizes. Now obviously if you're worried about attack, it's easy for someone to bypass this. So another option is to upload your files to S3 from the client side and setup a callback system to your Rails app. This keeps the traffic off of your main web servers and allows for you to process only the files you deem "safe".

Finally, if you choose to have user's upload to your server, you've mentioned limits that nginx & apahce give you for file size, heroku has a 30MB limit and 30sec timeout for their systems. If you're seeing repeat large size upload offenders and need to throttle the number of requests and/or ban users, you'll want to use Rack::Attack. I've used the gem a ton. It's simple to work with and is effective for what you're talking about.

The next level up from something like this is blocking at the Network level which you'll never get with Heroku, so you'll have to roll your own servers; and if we're talking network level security and attack mitigation, my recommendation is to hire a system admin that knows how to handle that!

As an aside, I'm happy that you're thinking security from the beginning, but designing for someone abusing a file upload feels like premature optimization (obviously for varying degrees of what type of app you're building).

Community
  • 1
  • 1
Gavin Miller
  • 43,168
  • 21
  • 122
  • 188
  • 1
    Thanks for the help Gavin! It's to mitigate risk for a live app. We're not going to be hiring or using experts. Security is an ongoing issue, but we have limited means and a modest yet tangible grasp of the problems. As you mention client-side validation is more of a UI/UX feature, but Heroku's request limits and your S3 suggestion help with that. Rack-attack is better than processing bad requests, but I'd rather the reverse-proxy drop them. Heroku doesn't expose a firewall or SDN API for this so I guess we're out of luck. I'm accepting your answer, as it was helpful and made me re-think. – nruth Nov 22 '15 at 17:12
  • so much for linebreaks in comments! – nruth Nov 22 '15 at 17:12
  • 1
    @nruth it is possible to setup a reverse-proxy on heroku using a nginx buildpack: https://github.com/ryandotsmith/nginx-buildpack. I've had limited success with this YMMV. – Gavin Miller Nov 22 '15 at 17:28
  • Good run through the options. This answer's quite old. I wonder if anything changed in rails 5/6/7. I was hoping rails had an easy way to tell it "Don't pre-parse my json, I want to do that in my controller logic after checking the body size". – Harry Wood Jul 13 '23 at 14:30