The crux of the issue should be that the VPN service only allows traffic that originates from the host machine (where the VPN client is running): other machines on the local network cannot reach the services behind the VPN.
Their network accessibility is limited to the same scope as the VPN service.
The VPN (Virtual Private Network) service itself, by its nature, tends to limit the network traffic to only go through the VPN tunnel for security reasons, making services behind it unreachable from the outside, unless specifically configured otherwise.
When a container uses another's network stack, it effectively becomes "hidden" behind that service. All incoming and outgoing network traffic for that container goes through the main service's network, which, in this case, is the VPN service. That is why you can access the vpn_img1 and vpn_img2 services from the host machine (where the VPN client is running), but not from other machines in the local network.
What you can try is to use a reverse proxy that can be accessed from the local network and forwards requests to the VPN-protected services.
Set up a new service in your Docker Compose file for the reverse proxy. You can use something like Nginx or Traefik. That service should not be behind the VPN.
Configure the reverse proxy to forward requests to vpn_img1 and vpn_img2. For Nginx, you might use the proxy_pass directive.
And ensure the proxy's ports are forwarded correctly, so you can access it from the local network.
Note that vpn_img1 and vpn_img2 are still behind the VPN, and any connections they make will go through it.
However, other services can reach them through the reverse proxy, which is not behind the VPN.
For instance, using NGiNX:
version: "3.4"
services:
vpn:
container_name: vpn
image: ghcr.io/bubuntux/nordlynx
restart: unless-stopped
cap_add:
- NET_ADMIN
environment:
- PRIVATE_KEY=xxx
- QUERY=filters\[country_id\]=228
- NET_LOCAL=192.168.1.0/24
sysctls:
- net.ipv6.conf.all.disable_ipv6=1
network_mode: bridge
vpn_img1:
container_name: vpn_img1
image: vpn_img1
restart: unless-stopped
network_mode: service:vpn
depends_on:
- vpn
vpn_img2:
container_name: vpn_img2
image: vpn_img2
restart: unless-stopped
network_mode: service:vpn
depends_on:
- vpn
host_img1:
container_name: host_img1
image: host_img1
restart: unless-stopped
network_mode: service:vpn # needs access to vpn_img2, so it needs to route through the VPN too
host_img2:
container_name: host_img2
image: host_img2
restart: unless-stopped
network_mode: host
nginx:
container_name: nginx
image: nginx
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf
ports:
- 8000:80
depends_on:
- vpn
network_mode: bridge
And, the nginx.conf file located in the same directory as your docker-compose.yml file, should be:
events {}
http {
server {
listen 80;
location /vpn_img1/ {
proxy_pass http://vpn_img1:1234/;
}
location /vpn_img2/ {
proxy_pass http://vpn_img2:2345/;
}
location /host_img1/ {
proxy_pass http://host_img1:3456/;
}
}
}
The nginx service is in bridge network mode so it can access the local network, and is also capable of reaching the vpn service since it is also in bridge mode. The proxy_pass directives are set to http://vpn:<port>/ because Nginx needs to send requests to the vpn service, which will forward them to the vpn_img services. <host_ip> is the IP address of the machine where Docker is running.
That means you should be able to reach vpn_img1 service at http://<host_ip>:8000/vpn_img1/, vpn_img2 service at http://<host_ip>:8000/vpn_img2/, and host_img1 service at http://<host_ip>:8000/host_img1/ from any machine in your local network.
Unfortunately, nginx fails to start, with the logs throwing host not found in upstream "vpn" in /etc/nginx/nginx.conf:8, which is where the first proxy_pass command is located (I get the same error for line 12, the next proxy_pass command, if I comment out the first location block).
I've ensured that both vpn and nginx have network_mode: bridge, and have tried adding vpn_img1 and vpn_img2 to nginx's depends_on section.
The error host not found in upstream "vpn" generally means that the Nginx container cannot resolve the hostname vpn to an IP address. In this case, it means that Nginx cannot find your vpn service.
That typically happens because the Docker internal DNS server is unable to resolve the service name to its IP address.
The network_mode: service:<name> is a bit special because it places the dependent service in the network stack of the service specified, not in a common network. That is why the nginx service cannot reach the vpn service directly, even if they are both in bridge network mode. The nginx service is in the default bridge network while the vpn service is isolated in its own network stack due to network_mode: service:vpn.
A partial solution may be to create a custom network and make sure all services are on that network, as suggested in this thread. That may still fail to fully resolve.
"no resolver defined to resolve vpn_img1" indicates that Nginx is unable to resolve the hostname vpn_img1 to an IP address. That is because Nginx does not use the system resolver, it uses its own DNS resolution implementation.
To address this, you can explicitly specify a DNS resolver in your Nginx configuration. That can be your local network's DNS server, or a public one like Google's DNS server at 8.8.8.8.
You should add the resolver directive in the http block, just before the server block, in the nginx.conf:
events {}
http {
resolver 127.0.0.11; # docker's internal DNS resolver
server {
listen 80;
location /vpn_img1/ {
proxy_pass http://vpn_img1:1234/;
}
location /vpn_img2/ {
proxy_pass http://vpn_img2:2345/;
}
location /host_img1/ {
proxy_pass http://host_img1:3456/;
}
}
}
In the above configuration, resolver 127.0.0.11 is used because Docker has a built-in DNS server that it runs at that IP address, which is used to provide DNS resolution for containers. That should help Nginx to resolve the container names to their respective IP addresses. Illustration/troubleshooting: "Docker Network Nginx Resolver".
See also "How to connect docker containers" from Adriano Galello for more on the Docker DNS service. And "docker-compose internal DNS server 127.0.0.11 connection refused" in case of issue.
Even files at / for a given location are getting a 404; so /index.html or /gettext.js return a 404. The service is instead requesting at a raw / location, so / when they should be asking at vpn_img1/.
Guess I need to bone up on my NGiNX. Any thoughts/tips?
The problem here could be related to how the applications you are proxying interpret the incoming requests. Many services that have been dockerized have a method for changing their internal pathing within the container's settings or in the docker configuration itself. If they don't, you can try to fix this in nginx itself.
For example, a request to <ip_address>:8000/vpn_img1/index.html would get forwarded to http://vpn_img1:1234/index.html.
Now, if your vpn_img1 service is set up such that it expects to be at the root (/), it may attempt to load resources from absolute paths (like /styles/main.css or /script/main.js). That might cause nginx to return a 404 because it is not configured to handle requests to these paths.
To work around this, you can strip off the path prefix (/vpn_img1 or /vpn_img2) from the request URI before the request is proxied to the respective services.
See also "Guide on how to use regex in Nginx location block section".
events {}
http {
server {
listen 80;
location ~ ^/vpn_img1/(.*)$ {
proxy_pass http://vpn_img1:1234/$1;
}
location ~ ^/vpn_img2/(.*)$ {
proxy_pass http://vpn_img2:2345/$1;
}
location ~ ^/host_img1/(.*)$ {
proxy_pass http://host_img1:3456/$1;
}
}
}
Here, the regular expression matches the location, and any path after /vpn_img1/ or /vpn_img2/ is captured into the $1 variable. That value is then used in the proxy_pass directive. Now, a request to <ip_address>:8000/vpn_img1/index.html would be forwarded to http://vpn_img1:1234/index.html, and any resources requested by index.html from the root will also be correctly proxied to http://vpn_img1:1234/.
That should work as long as the applications served by vpn_img1 and vpn_img2 use relative paths to refer to other resources within the application. If they use absolute paths (for example, /styles/main.css instead of styles/main.css), then there might still be issues, and you would need to modify your applications to use relative paths instead.
The OP confirms in the comments:
Adding that resolver and rewrite /vpn_app1(.*) $1 break; proxy_set_header <hostname> "/vpn_app1/"; in the location block (as per documentation) did it!
Your final Nginx configuration would look something like this:
events {}
http {
resolver 127.0.0.11;
server {
listen 80;
# for a service that had an internal method for path redirection
location ^~ /vpn_img1/ {
proxy_pass http://vpn_img1:1234;
}
# for a service without (trickier to get right)
location ~ ^/vpn_img2/(.*)$ {
rewrite /vpn_img2(.*) $1 break;
proxy_set_header Host $host;
proxy_pass http://vpn_img2:2345;
}
# each service may need additional tweaking
location ^~ /host_img1/ {
proxy_set_header Host $host;
proxy_pass http://host_img1:3456;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection keep-alive;
proxy_cache_bypass $http_upgrade;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $http_host;
}
}
}
With this configuration, the incoming requests to /vpn_img1/... and /vpn_img2/... are being rewritten to /... before being passed to the corresponding services. And the proxy_set_header Host $host; line makes sure that the Host header from the incoming HTTP request is forwarded to the proxied server.
