Nginx client_max_body_size not working in Docker container on AWS Elastic Beanstalk

Question

I'm having a problem where nginx seems to be ignoring (or overriding) my upped client_max_body_size directive in a Ubuntu Docker container on AWS Elastic Beanstalk. This is preventing users from uploading files any larger than the nginx default of 1MB.

I have used the client_max_body_size 10M; in http, server,and location blocks to no avail, I am still seeing "client intended to send too large body" errors in the nginx logs. I have successfully used these settings on an AWS EC2 Ubuntu instance, but since using the same setup in a Docker container I am having this problem. I've also tried using an ebextension as outlined here Increasing client_max_body_size in Nginx conf on AWS Elastic Beanstalk

The app itself is CFML (Railo) running in a Tomcat container.

Here are the relevant nginx files:

The full unabridged files are here https://github.com/chapmandu/docker-railo

Thanks in advance.

nginx error.log

2014/12/02 03:02:05 [error] 32116#0: *142 client intended to send too large body: 1290803 bytes, client: 172.31.19.39, server: , request: "POST /listings/35602/images/create HTTP/1.1", host: "staging.svr.com.au", referrer: "http://staging.svr.com.au/listings/35602/images/new"

nginx.conf

daemon off;

worker_processes  1;

events {
    worker_connections  1024;
}

http {
    client_max_body_size 10M;
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    include /etc/nginx/sites-enabled/default;
}

default

server
{
    listen       80;
    server_name  localhost;

    client_max_body_size 10M;

    # don't rewrite for static files
    location ~* ^.+\.(jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|pdf|txt|tar|wav|bmp|rtf|js|flv|swf|html|htm|map|ttf|woff)$
    {
        root   /var/www;
    }

    location /
    {
        root   /var/www;
        index  index.cfm;
        client_max_body_size 10M;
        include proxy_params;
    }
}

proxy_params

proxy_redirect off;

# # If you want your server to identify itself only as Tomcat you can pass
# # the Tomcat setting to Nginx telling Nginx not to change it
#proxy_pass_header Server;

# Point Nginx to Tomcat
proxy_pass  http://localhost:8080;

# Send appropriate headers through
# Forward the real ip to Tomcat (and Railo)

proxy_buffers 16 16k;
proxy_buffer_size 32k;

# prevent regular 504 Gateway Time-out message
proxy_connect_timeout       600;
proxy_send_timeout          600;
proxy_read_timeout          600;
send_timeout                600;

# pass headers through
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Query-String $request_uri;
proxy_set_header X-Host $host;
proxy_set_header X-Remote-Addr $remote_addr;
proxy_set_header X-Request-Filename $request_filename;
proxy_set_header X-Request-URI $request_uri;
proxy_set_header X-Server-Name $server_name;
proxy_set_header X-Server-Port $server_port;
proxy_set_header X-Server-Protocol $server_protocol;

proxy_intercept_errors on;

# apparently this is how to disable cache?
expires     -1;

chapmandu · Accepted Answer · 2014-12-10T00:09:33.510

10

It turns out that AWS uses nginx to proxy connections to the docker container, it's necessary to update the AWS nginx configuration in order to set client_max_body_size. Note that I am using 64bit Amazon Linux 2014.09 v1.0.9 running Docker 1.2.0.

I needed to create .ebextensions/01-client-max-body.config with the contents below.

The important line is "server 127.0.0.1:EB_CONFIG_NGINX_UPSTREAM_PORT;"

files:
  "/tmp/custom-nginx.conf":
    mode: "00644"
    owner: "root"
    group: "root"
    content: |
      upstream docker {
        server 127.0.0.1:EB_CONFIG_NGINX_UPSTREAM_PORT;
        keepalive 256;
      }

      server {
        listen EB_CONFIG_HTTP_PORT;
        client_max_body_size 10M;

        location / {
          proxy_pass      http://docker;
          proxy_http_version      1.1;
          proxy_set_header Host                $host;
          proxy_set_header X-Real-IP           $remote_addr;
          proxy_set_header X-Forwarded-For     $proxy_add_x_forwarded_for;
        }
      }

container_commands:
  01_remove_orig_config:
    command: 'sed -i ''/EOF/,/EOF/d'' /opt/elasticbeanstalk/hooks/appdeploy/enact/00flip.sh '
  02_add_new_conf:
    command: 'sed -i ''/# set up nginx/a sed -i s\/EB_CONFIG_NGINX_UPSTREAM_PORT\/$EB_CONFIG_NGINX_UPSTREAM_PORT\/ \/tmp\/custom-nginx.conf \nsed -i s\/EB_CONFIG_HTTP_PORT\/$EB_CONFIG_HTTP_PORT\/ \/tmp\/custom-nginx.conf \ncat \/tmp\/custom-nginx.conf > \/etc\/nginx\/sites-available\/elasticbeanstalk-nginx-docker.conf'' /opt/elasticbeanstalk/hooks/appdeploy/enact/00flip.sh'
    test: 'test ! $(grep custom-nginx.conf /opt/elasticbeanstalk/hooks/appdeploy/enact/00flip.sh)'

Answer provided by super helpful AWS support..

edited Dec 10 '14 at 00:09

answered Dec 07 '14 at 23:11

chapmandu

266
1
2
8

Here is an answer to a related question that worked for me when using the python environment on beanstalk. Not sure if it works on docker environments as well, but the solution is simpler, it just adds another file to the conf.d directory: http://stackoverflow.com/a/18951706/933245 – yellowcap Apr 03 '15 at 12:38
Cheers, the solution I posted was for Docker 1.2. The solution I received from AWS supprt for Docker 1.3+ is pretty much exactly as per your link. – chapmandu Apr 05 '15 at 10:37
1

I see that nginx configuration is quite different from what reported above, and resetting the whole upstream proxy seems an overkill (plus, it's already defined in `/etc/nginx/conf.d/elasticbeanstalk-nginx-docker-upstream.conf`) if it's used just to increase the client body size. this answer: http://stackoverflow.com/a/28313384/515153 should do the trick – gru Oct 19 '15 at 13:37
this is the only way I've managed to make it work using docker on Elastic Beanstalk. – gcw Jun 17 '16 at 16:51
I tried this and it still did not fix the issue for me..... i'm running out of ideas – digitaldavenyc May 07 '17 at 18:05
This answer didn't work for me as I don't have this `/00flip.sh` on my instance. See my answer below for what ended up working – Sam Borick Jan 04 '18 at 21:22

score 10 · Answer 2 · answered Jan 04 '18 at 21:20

The accepted answer didn't work for me, but I managed to figure it out.

Previously I used a prebuilt docker image (hosted in my own registry), and specified access and url in the dockerrun.aws.json file, and then had a config file at .elasticbeanstalk/config.json with the following:

deploy:
  artifact: Dockerrun.aws.json

This uploaded only the dockerrun file as an artifact.

In order to set the client_max_body_size via ebextentions, you must switch to either uploading the entire folder via eb deploy or specify a zip file artifact containing your .ebextentions folder

I then had success with a config file at .ebextentions/size.config containing:

files:
  /etc/nginx/conf.d/proxy.conf:
    content: |
      client_max_body_size 50M;
container_commands:
   01_reload_nginx:
     command: "service nginx reload"

I arrived at this answer thanks to https://stackoverflow.com/a/23587371/2653503, https://stackoverflow.com/a/39829789/2653503, and a few others that I can't find right now.

To clarify on this, all you have to do is include the .ebextensions folder in the source bundle zip together with the dockerrun json file. Then you can still use the prebuilt docker image as before. — Adam, Aug 22 '20 at 01:09

score 1 · Answer 3 · answered Dec 03 '14 at 10:51

1

A few stabs in the dark ... I notice you have 2 occurrences of the client_max_body_size parameter in your example, and that your github configuration is different. If there is more than one occurrence of a directive in a config file or include, nginx will take the value of the directive defined last.

Another possibility to consider is that files uploaded via a web form will often have a larger file size than that shown in the file system. Have you tried a extra large setting, such as

client_max_body_size 500m;

Reference: Mastering Nginx

answered Dec 03 '14 at 10:51

Nando

94
5

I've tried all combos of the client_max_body_size directive. The one posted above is simply the most recent. I currently have the limit at 10mb but I can't upload anything (even slightly) above 1000kb. I don't really want to increase the limit to something like 500M as I actually want to limit uploads to 10M. – chapmandu Dec 03 '14 at 21:42

score 0 · Answer 4 · answered May 15 '19 at 22:05

Here's another option I ran into, apparently, modify the nginx.conf inline: create this ebextensions file:

commands:
  01_filesize:
    cwd: /etc/nginx
    test: "! grep client_max_body_size nginx.conf"
    command: sed -i -e 's/http {/http {\n    client_max_body_size 0;\n/' nginx.conf
  02_filesize:
    cwd: /etc/nginx/sites-available
    test: "! grep client_max_body_size elasticbeanstalk-nginx-docker-proxy.conf"
    command: sed -i -e 's/server {/server {\n        client_max_body_size 0;\n/'  elasticbeanstalk-nginx-docker-proxy.conf;sed -i -e 's/location \/ {/location \/ {\n            client_max_body_size 0;\n/' elasticbeanstalk-nginx-docker-proxy.conf

Nginx client_max_body_size not working in Docker container on AWS Elastic Beanstalk

nginx error.log

nginx.conf

default

proxy_params

4 Answers4

Linked