Bad Gateway for Nginx Proxy Manager in Nerdctl when target is reachable from proxy container

Question

I have Nginx Poxy Manager and a helloworld-http container running in the same bridged network.

A proxy host has been setup for some domain to hit the proxy server, and it's configured to hit the hello server inside the bridged network.

Inside the container running the proxy, the target is reachable and the response confirms the system name:

While the generated nginx config for the proxy looks to be correct as well:

/data/nginx/proxy_host/2.conf

server {
  set $forward_scheme http;
  set $server         "hello";
  set $port           80;

  listen 80;
listen [::]:80;


  server_name somedomainbeingused.com;

I found this question stating the host has to have an index.html and confirmed the test server will respond to that as well

I've even checked that the browser is not auto updating the protocol to https.

All the stars here seem to line up so, what am I missing?

EDIT: upon further review i've checked the logs and it seems curious.

0000/00/00 00:00:00 [error] 97162#97162: *1575 hello could not be resolved (3: H
ost not found), client: 10.4.2.1, server: somedomainbeingused.com, request: "GET 
/ HTTP/1.1", host: "somedomainbeingused.com"

What doesn't make sense about the error is i've confirmed the host (hello) is reachable from the container hosting the proxy (where the log was generated).

Answer 1

So, the reason this was not working is because of the way networking works in Docker / Nerdctl. For all bridge networks there is an internal DNS resolver. In docker this is typically 127.0.0.11 but regardless it's written to /etc/resolv.conf in the container for the system and services to use this resolver. Ngingx does this, however in Nerdctl (which is what's managing my containers) the resolver only resolves dns external to the network ... ... ... This is transparent in most cases because Nerdctl writes the full network configuration to the hosts file. So, when you curl, hosts is used first instead of an external resolver, unless you're Nginx ... ... ... and you use the resolver exclusively.

(context for the explanation)

So the solution seems to be install Dnsmasq to read from hosts for Nginx and set it in /etc/resolv.conf.