Unable to connect to redis installed on my development host machine from development container

Question

I have a redis cluster running on my MacBook (localhost), I am easing my way into docker as part of my developer environment.

I also have things like MongoDB running on my host and am able to successfully connect to via host.docker.internal, at this time i'm not looking to containerize redis, or my other services

Im specifically getting errors that my server is somehow trying to connect using 127.0.0.1 when my code is definitely using host.docker.internal

this is happening on the current nodejs container, but also another ruby container - so something about redis is unhappy

oauth  | 2022-07-22T21:57:58.186Z app:cache:client FATAL Redis Cluster Error Error: connect ECONNREFUSED 127.0.0.1:30003
oauth  | 2022-07-22T21:57:58.188Z app:cache:client FATAL Redis Cluster Error Error: connect ECONNREFUSED 127.0.0.1:30001
oauth  | 2022-07-22T21:57:58.189Z app:cache:client FATAL Redis Cluster Error Error: connect ECONNREFUSED 127.0.0.1:30002

Dockerfile

# syntax=docker/dockerfile:1
FROM node:16.16.0-buster as base

WORKDIR /app

COPY package.json package.json
COPY yarn.lock yarn.lock

FROM base as dev
RUN yarn install
COPY . .
CMD ["node", "src/index.js"]

docker-compose.dev.yml

version: '3.8'
services:
  oauth:
    build:
      context: .
    container_name: oauth
    ports:
      - 5050:5050
    environment:
      - RACK_ENV=docker
      - PORT=5050
    volumes:
      - ./:/app
    command: yarn run nodemon ./src/index.js

docker-compose -f docker-compose.dev.yml up --build
[+] Building 6.8s (17/17) FINISHED                                                                                                                                                                                                                                              
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                       0.0s
 => => transferring dockerfile: 245B                                                                                                                                                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                                                                                                                                            0.0s
 => resolve image config for docker.io/docker/dockerfile:1                                                                                                                                                                                                                 0.8s
 => [auth] docker/dockerfile:pull token for registry-1.docker.io                                                                                                                                                                                                           0.0s
 => CACHED docker-image://docker.io/docker/dockerfile:1@sha256:443aab4ca21183e069e7d8b2dc68006594f40bddf1b15bbd83f5137bd93e80e2                                                                                                                                            0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                          0.0s
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                       0.0s
 => [internal] load metadata for docker.io/library/node:16.16.0-buster                                                                                                                                                                                                     0.5s
 => [auth] library/node:pull token for registry-1.docker.io                                                                                                                                                                                                                0.0s
 => [internal] load build context                                                                                                                                                                                                                                          0.9s
 => => transferring context: 3.52MB                                                                                                                                                                                                                                        0.9s
 => [base 1/4] FROM docker.io/library/node:16.16.0-buster@sha256:2e1b4542d4a06e0e0442dc38af1f4828760aecc9db2b95e7df87f573640d98cd                                                                                                                                          0.0s
 => CACHED [base 2/4] WORKDIR /app                                                                                                                                                                                                                                         0.0s
 => CACHED [base 3/4] COPY package.json package.json                                                                                                                                                                                                                       0.0s
 => CACHED [base 4/4] COPY yarn.lock yarn.lock                                                                                                                                                                                                                             0.0s
 => CACHED [dev 1/2] RUN yarn install                                                                                                                                                                                                                                      0.0s
 => [dev 2/2] COPY . .                                                                                                                                                                                                                                                     2.2s
 => exporting to image                                                                                                                                                                                                                                                     2.0s
 => => exporting layers                                                                                                                                                                                                                                                    2.0s
 => => writing image sha256:a297a1937c12a7403b12dec58a71c28df60caa0ee387daec51af2ffb0dc5968e                                                                                                                                                                               0.0s
 => => naming to docker.io/library/oauth_oauth                                                                                                                                                                                                                             0.0s
[+] Running 1/1
 ⠿ Container oauth  Recreated                                                                                                                                                                                                                                              0.1s
Attaching to oauth
oauth  | yarn run v1.22.19
oauth  | $ NODE_ENV=docker yarn run nodemon ./src/index.js
oauth  | $ /app/node_modules/.bin/nodemon ./src/index.js
oauth  | [nodemon] 2.0.16
oauth  | [nodemon] to restart at any time, enter `rs`
oauth  | [nodemon] watching path(s): *.*
oauth  | [nodemon] watching extensions: js,mjs,json
oauth  | [nodemon] starting `node ./src/index.js`
oauth  | 2022-07-22T21:57:58.161Z app:index INFO ⚡️ Successfully Started Express Server
oauth  | 2022-07-22T21:57:58.162Z app:index INFO ⚡️ Environment: docker
oauth  | 2022-07-22T21:57:58.163Z app:index INFO ⚡️ Node Version: v16.16.0
oauth  | 2022-07-22T21:57:58.163Z app:index INFO ⚡️ Listening on: http://localhost:5050
oauth  | 2022-07-22T21:57:58.163Z app:index INFO ⚡️ OS linux
oauth  | 2022-07-22T21:57:58.186Z app:cache:client FATAL Redis Cluster Error Error: connect ECONNREFUSED 127.0.0.1:30003
oauth  | 2022-07-22T21:57:58.188Z app:cache:client FATAL Redis Cluster Error Error: connect ECONNREFUSED 127.0.0.1:30001
oauth  | 2022-07-22T21:57:58.189Z app:cache:client FATAL Redis Cluster Error Error: connect ECONNREFUSED 127.0.0.1:30002

More info:

Redis Cluster is ran by doing the following: (i've been using this for several years, it works) https://developer.redis.com/explore/redisinsight/cluster/

$ cd ~/Documents/dev/redis-6.2.6; ./utils/create-cluster/create-cluster start;
Starting 30001
Starting 30002
Starting 30003
Starting 30004
Starting 30005
Starting 30006
$ redis-cli -c -p 30001
127.0.0.1:30001> ping
PONG

i connected to the container to prove that host.docker.internal resolves to host

$ docker exec -it <container id> /bin/bash
$ ping host.docker.internal
PING host.docker.internal (192.168.65.2) 56(84) bytes of data.
64 bytes from 192.168.65.2 (192.168.65.2): icmp_seq=1 ttl=37 time=0.182 ms
64 bytes from 192.168.65.2 (192.168.65.2): icmp_seq=2 ttl=37 time=0.275 ms
64 bytes from 192.168.65.2 (192.168.65.2): icmp_seq=3 ttl=37 time=0.230 ms
64 bytes from 192.168.65.2 (192.168.65.2): icmp_seq=4 ttl=37 time=0.516 ms
64 bytes from 192.168.65.2 (192.168.65.2): icmp_seq=5 ttl=37 time=0.540 ms
64 bytes from 192.168.65.2 (192.168.65.2): icmp_seq=6 ttl=37 time=0.560 ms
64 bytes from 192.168.65.2 (192.168.65.2): icmp_seq=7 ttl=37 time=0.566 ms
^C
--- host.docker.internal ping statistics ---
7 packets transmitted, 7 received, 0% packet loss, time 138ms
rtt min/avg/max/mdev = 0.182/0.409/0.566/0.161 ms

However, netstat doesnt return anything for 30001 only 30003

$ netstat -a | grep 30001
(nothing)

$ netstat -a | grep 30003
tcp4       0      0  localhost.30003        localhost.51793        ESTABLISHED
tcp4       0      0  localhost.51793        localhost.30003        ESTABLISHED
tcp6       0      0  *.30003                *.*                    LISTEN     
tcp4       0      0  *.30003                *.*                    LISTEN

the default config for redis when it starts is here

# Settings
BIN_PATH="/Users/aronlilland/Documents/dev/redis-6.2.6/src"
CLUSTER_HOST=127.0.0.1
PORT=30000
TIMEOUT=2000
NODES=6
REPLICAS=1
PROTECTED_MODE=yes
ADDITIONAL_OPTIONS=""

# You may want to put the above config parameters into config.sh in order to
# override the defaults without modifying this script.

if [ -a config.sh ]
then
    source "config.sh"
fi

# Computed vars
ENDPORT=$((PORT+NODES))

if [ "$1" == "start" ]
then
    while [ $((PORT < ENDPORT)) != "0" ]; do
        PORT=$((PORT+1))
        echo "Starting $PORT"
        $BIN_PATH/redis-server --port $PORT  --protected-mode $PROTECTED_MODE --cluster-enabled yes --cluster-config-file nodes-${PORT}.conf --cluster-node-timeout $TIMEOUT --appendonly yes --appendfilename appendonly-${PORT}.aof --dbfilename dump-${PORT}.rdb --logfile ${PORT}.log --daemonize yes ${ADDITIONAL_OPTIONS}
    done
    exit 0
fi

Did you comment out this line in your `redis.conf` file and restart? `# bind 127.0.0.1 ::1` — Mark Setchell, Jul 22 '22 at 23:02
You should be able to see if Redis is listening on all interfaces (i.e. 0.0.0.0) or just localhost (i.e. 127.0.0.1) with `sudo netstat -tunlp` — Mark Setchell, Jul 22 '22 at 23:05
If you run the `netstat` command I suggested, is **Redis** listening on 0.0.0.0 port 6379 ? — Mark Setchell, Jul 22 '22 at 23:45
If you are trying to connect to **Redis** running on your Mac from within a docker container, it will need to use the IP address of your Mac, not 127.0.0.1 which is shorthand for *"the local machine"*, i.e. something inside the container. — Mark Setchell, Jul 22 '22 at 23:48
redis is not listening on port 6379, its running in cluster mode on `30001`, `30002`, `30003`, `30004`, `30005`, `30006` -- ive been using it for several years and it works while not running in the container, so its absolutely on. running `netstat -a | grep 30003` returns several established connections on tcp4, and 2 listening connections on tcp6 - other systems are able to connect using `host.docker.internal` instead of localhost, its redis that doing something weird — alilland, Jul 23 '22 at 04:41
Redis is running on 127.0.0.1 which means it will not allow connections from docker containers or from any other machines on your network. It needs to listen on **all interfaces** for connections, i.e. on 0.0.0.0. Both `netstat` and `redis-cli` are telling you Redis is on 127.0.0.1 — Mark Setchell, Jul 23 '22 at 05:33
Same as here... https://stackoverflow.com/a/72882529/2836621 — Mark Setchell, Jul 23 '22 at 05:37
my mongodb instance is listening on localhost:27017 and docker is able to connect to it without it being bound to 0.0.0.0 - binding to 0.0.0.0 is highly vulnerable https://stackoverflow.com/questions/60572003/redis-cluster-create-replicas-bind-public-ip there is no reason for my entire home network/work network to see redis as an open port, the purpose of docker for dev is to make a local network that doesn't leave the box — alilland, Jul 23 '22 at 05:44
after running some tests by installing the redis-cli in the container, i confirmed that the container is in fact able to reach the host redis nodes using `host.docker.internal`, what I believe is happening is that the redis clusters themselves are responding back with their localhost master/slave url's which are `127.0.0.1` so when the nodejs client tries to use the url provided from the redis response, it fails — alilland, Jul 25 '22 at 16:25

Unable to connect to redis installed on my development host machine from development container

Dockerfile

docker-compose.dev.yml

0 Answers0