1

I have a docker application running on a server A (let's call it appA) and a dockerized PostgreSQL database running on a server B (along with an appB).

I need to connect, from the appA container which is on server A to the database on server B.

Server B is only accessible using an SSH tunnel.

The tunnel is working fine when I'm directly on the server A (not inside the container) and I can connect to the db using, e.g.:

userA@serverA$ ssh -f -N -L 5433:127.0.0.1:5432 userB@serverB
userA@serverA$ psql -d "postgres://dbuser:dbpassword@localhost:5433/mydatabase"

mydatabase=> \dt (prints my relations)

But when executed in the appA container, the exact same commands says:

root@057da644917e:/appA# psql -d "postgres://dbuser:dbpassword@localhost:5433/mydatabase"
psql: error: could not connect to server: Connection refused
    Is the server running on host "localhost" (127.0.0.1) and accepting
    TCP/IP connections on port 5433?

I suspect that localhost inside the docker container is not the same as on the server A.
Then I went there: access host's ssh tunnel from docker container

And I tried to open the SSH tunnel as follow from the server A:

$ ssh -f -N -L 172.17.0.1:5433:127.0.0.1:5432 userB@serverB

where 172.17.0.1 is the value of "inet addr" in the docker0 block when executing ifconfig.

But then, from the appA container, the connection seems to time out:

root@057da644917e:/appA# psql -d "postgres://dbuser:dbpassword@172.17.0.1:5433/mydatabase"
psql: error: connection to server at "172.17.0.1", port 5433 failed: Connection timed out
    Is the server running on that host and accepting TCP/IP connections?

(again, that same command is working perfectly fine when executed directly on the server A)

What am I missing?

I know the host.docker.internal on Windows but here, both server A and B are on Ubuntu and I cannot find an equivalent trick for Linux.

Also, network: host is not an option.

Info:

Docker version 20.10.11, build dea9396    
docker-compose version 1.27.4, build 40524192

userA@serverA$ docker ps -a
CONTAINER ID   IMAGE               COMMAND     CREATED          STATUS          PORTS       NAMES
0da42f402a21   myapp/tag:0.0.1     "python3"   29 seconds ago   Up 29 seconds   5001/tcp    appfolder_app_1

This is the output of netstat from the host:

userA@serverA$ netstat -tulnp | grep 5433 
tcp   0   0   172.17.0.1:5433   0.0.0.0:*   LISTEN   44828/ssh

I can also successfully ping the Gateway from inside the appA container on server A:

root@057da644917e:/app# ping -c4 172.17.0.1
PING 172.17.0.1 (172.17.0.1) 56(84) bytes of data.
64 bytes from 172.17.0.1: icmp_seq=1 ttl=64 time=0.073 ms
64 bytes from 172.17.0.1: icmp_seq=2 ttl=64 time=0.049 ms
64 bytes from 172.17.0.1: icmp_seq=3 ttl=64 time=0.048 ms
64 bytes from 172.17.0.1: icmp_seq=4 ttl=64 time=0.049 ms

--- 172.17.0.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3050ms
rtt min/avg/max/mdev = 0.048/0.054/0.073/0.010 ms

I narrowed down the timeout issue to this, this time only on server A (so no tunnel or connection to server B; I only want to reach the local PostgreSQL database, which does exist):

userA@serverA$ docker run -it --rm --name postgis postgis/postgis:14-3.1 bash

root@7db073ef354a:/# psql -U postgres -h 172.17.0.1 -p 5432
psql: error: connection to server at "172.17.0.1", port 5432 failed: Connection timed out
    Is the server running on that host and accepting TCP/IP connections?

docker inspect on this postgis container gives NetworkSettings.Networks.bridge.Gateway: 172.17.0.1.

swiss_knight
  • 5,787
  • 8
  • 50
  • 92
  • I took my answer down, since it didn't help much. Couple of other thing I'd try at this point is - verify iptables on the host, try a different/empty/unmodified container and lastly i'd start capturing traffic on both Server A (`sudo tcpdump -ns0 -i docker0 port 5433`) and Server B (`sudo tcpdump -ns0 -i lo port 5432`). This has to be setup related, as I am not able to repro, for me it just works. In my tests I used ubuntu everywhere (server a, server b, app container). – jabbson Dec 04 '21 at 20:42
  • Ok, I'm still investigating more. But from both commands you send (thanks!) only the one on server B shows some output in the case of a direct connection (i.e. not from the container). And it's showing output either when using `localhost` or the gateway `172.17.0.1` when used as the bind address. Absolutely no output is shown when I try from the container. – swiss_knight Dec 05 '21 at 08:25
  • Is it docker0 interface that has the IP address 172.17.0.1 on the host? What if you run `sudo tcpdump -ns0 -i docker0 icmp`, do you capture pings? What about these iptable rules? – jabbson Dec 05 '21 at 15:02

0 Answers0