nginx reverse proxy multiple host configuration files instead of one big file

Question

link to issue

I have still a question. Is it possible, to have for each system I reverse proxy like above, to have a separate conf file in the modules-enabled folder. I tried this but, nginx keeps saying, that the stream directive is duplicate. Maybe I can point the one stream module to various server confs, but I don't know how to do that.

Could anyone help?

Answer 1

I played a bit with your use case in my lab environment and you actually found a pretty neat solution to stream proxy pass depending on an incoming subdomain. The main problem is still as mentioned in my comment: you must not have two times the same port you are listening to (in your case 443). That is something you don't have in the big combined file, but you are introducing it in the divided files and as nginx is simply concatenating the separated files the error you mention occurs.

From my point of view by changing the splitting of the files it should work. Here an example I find useful. It consists of 3 parts:

• a "prefix" part which contains the mapping
• the servers, add as much as you like. One server has a separate server {} directive with port 5480 as this one is only used once it can go in the server specific file. If that would occur also for other servers, it must go in the suffix file.
• a "suffix" part which contains the actual proxy pass directive which is always the same due to the usage of the $upstreamvariable

I started the filenames with numbers to ensure their order when nginx concatenates them.

Please note that I have tested this with DNS & servers I have and can control. I did rewrite the examples so that they match your situations. I hope it's working as is, but as I don't have your setup it might not work right away.

Prefix: 00_prefix.conf

map $ssl_preread_server_name:$server_port $upstream {
  tempserver01.domain.com:5480 tempserver01_vapp_5480;
  tempserver01.domain.com:443 tempserver01_backend_443;
  tempserver02.domain.com:443 tempserver02_backend_443;
}

Server 1: 01_realserver01.conf

upstream $upstream {
  hash $remote_addr consistent;
  server duckduckgo.com:443;
}

server {
  listen 5480;
  proxy_pass $upstream;
  ssl_preread on;
}

Server 2: 02_realserver02.conf

upstream $upstream {
  hash $remote_addr consistent;
  server duckduckgo.com:443;
}

You for sure have realized that the server {} part was missing. That’s due to the fact that it is 100% the same in all cases and thus must be separated in one file to ensure that it ends up only 1 time in the concatenated big config nginx is creating out of all the bits and pieces in the separated config files.

Suffix: 99_suffix.conf

server {
  listen 443;
  proxy_pass $upstream;
  ssl_preread on;
}

After having reloaded nginx with nginx -s reload you can check the complete concatinated config nginx has loaded with nginx -T and you'll see something like this:

# configuration file /etc/nginx/stream/enabled/00_prefix.conf:
map $ssl_preread_server_name:$server_port $upstream {
  tempserver01.domain.com:5480 tempserver01_vapp_5480;
  tempserver01.domain.com:443 tempserver01_backend_443;
  tempserver02.domain.com:443 tempserver02_backend_443;
}
# configuration file /etc/nginx/stream/enabled/01_realserver01.conf:
upstream $upstream {
  hash $remote_addr consistent;
  server duckduckgo.com:443;
}

server {
  listen 5480;
  proxy_pass $upstream;
  ssl_preread on;
}
# configuration file /etc/nginx/stream/enabled/02_realserver02.conf:
upstream $upstream {
  hash $remote_addr consistent;
  server duckduckgo.com:443;
}
# configuration file /etc/nginx/stream/enabled/99_suffix.conf:
server {
  listen 443;
  proxy_pass $upstream;
  ssl_preread on;
}

As you can see the separated files are combined in the right order and everything should work as expected. I hope that helps you.