8

With the advent of Docker 20.10, host-gateway is supposed to be available on Linux platforms (as detailed in this wonderful answer). As such, it should be possible to create a docker-compose script which is platform agnostic. (I myself am on Debian.)

Here are some links to some questions and answers that were helpful in getting me this far: here, here, and here (along with some of the other answers and comments)

I'm trying to create a script for running The Graph, which involves having ipfs and postgres running inside a Docker container, and connecting to an instance of a blockchain outside of Docker (on port 8545). Here is the script:

 version: '3'
services:
  graph-node:
    extra_hosts:
      - "host.docker.internal:host-gateway"
    image: graphprotocol/graph-node
    ports:
      - '8000:8000'
      - '8001:8001'
      - '8020:8020'
      - '8030:8030'
      - '8040:8040'
    depends_on:
      - ipfs
      - postgres
    environment:
      postgres_host: postgres
      postgres_user: graph-node
      postgres_pass: let-me-in
      postgres_db: graph-node
      ipfs: 'ipfs:5001'
      ethereum: 'localhost:http://host.docker.internal:8545'
      RUST_LOG: info
  ipfs:
    image: ipfs/go-ipfs:v0.4.23
    ports:
      - '5001:5001'
    volumes:
      - ./data/ipfs:/data/ipfs
  postgres:
    image: postgres
    ports:
      - '5432:5432'
    command: ["postgres", "-cshared_preload_libraries=pg_stat_statements"]
    environment:
      POSTGRES_USER: graph-node
      POSTGRES_PASSWORD: let-me-in
      POSTGRES_DB: graph-node
    volumes:
      - ./data/postgres:/var/lib/postgresql/data

Docker starts just fine, and the instances of ipfs, postgres, and the graph-node all start up fine, but then the graph-node's RPC calls (to the blockchain) all fail with errors similar to the following:

WARN Trying again after eth_getBlockByNumber(0, false) RPC call failed (attempt #18) with result Err(Transport error: Error(Connect, Os { code: 111, kind: ConnectionRefused, message: "Connection refused" }))

Am I using extra-hosts wrong? What might I be able to do to make this script work both on my Linux machine, but also for Mac and Windows users?

Thanks!

The Renaissance
  • 431
  • 9
  • 16
  • Did you figure out a solution for that problem? I have the same issue.. – Ndrslmpk Feb 02 '21 at 14:06
  • @Twixx unfortunately not – The Renaissance Feb 02 '21 at 14:26
  • 1
    I am running into this issue now. it just randomly started happening. When I ssh into the graph-node docker, I can ping into the host no problem, but as soon as I add the port 8545 to connect to chain I get connection refused. I am using --hostname 0.0.0.0 option on the hardhat node, so not sure what can be going on here.... – Naz May 19 '21 at 22:51
  • 1
    hmm and now it randomly works... seems like it was this --hostname 0.0.0.0 option on the hardhat node that fixed it. – Naz May 19 '21 at 22:53

2 Answers2

3

It is possible that the application running on your host is bound to a different interface than the one used by Docker.

You can check with netstat:

$ netstat -pan | grep 8545
tcp6       0      0 127.0.0.1:8545          :::*                    LISTEN      496150/java

If it is listening on 127.0.0.1, like in this example, this means it can only be accessed via the loopback interface.

The solution is to find out what IP address host-gateway points to and make sure the service binds to that IP address instead of 127.0.0.1.

If it's no problem when the service is available on all interfaces (including for example your wifi network), you can bind to 0.0.0.0 to make it available on all interfaces.

Arnout Engelen
  • 6,709
  • 1
  • 25
  • 36
  • I'm running into this issue as well and I see the following on the host when I run the `netstat` command: `tcp6 0 0 :::8545 :::* LISTEN 23703/geth`. – Paymahn Moghadasian Mar 09 '22 at 21:22
  • 1
    This means your service is already listening on all interfaces (`::` is the IPv6-compatible way of writing `0.0.0.0`), so you don't have the problem for which my answer is the solution. I know of another possible problem that might match the same symptoms, though - I'll add a second answer about that. – Arnout Engelen Mar 10 '22 at 08:53
1

If the host system is Linux, it is possible that your host firewall rules are being applied to communication between your containers. You can check whether that is the case with sysctl, it would return 1 for the following settings:

$ sysctl net.bridge.bridge-nf-call-arptables
1
$ sysctl net.bridge.bridge-nf-call-iptables
1
$ sysctl net.bridge.bridge-nf-call-ip6tables

You can fix this behavior by setting these values to '0':

$ sudo sysctl net.bridge.bridge-nf-call-arptables=0
net.bridge.bridge-nf-call-arptables = 0
$ sudo sysctl net.bridge.bridge-nf-call-iptables=0
net.bridge.bridge-nf-call-iptables = 0
$ sudo sysctl net.bridge.bridge-nf-call-ip6tables=0
net.bridge.bridge-nf-call-ip6tables = 0
Arnout Engelen
  • 6,709
  • 1
  • 25
  • 36
  • Yup, all of those values were `1`. I changed them all to `0` but still when I run a container and then `exec -it /bin/bash` into it, I can't connect to the host on port 8545: `root@localhost:~/code/metis/ops# docker exec -it ops_l2geth-mainnet_1 /bin/sh / # geth attach http://host.docker.internal:8545 Fatal: Failed to start the JavaScript console: api modules: Post "http://host.docker.internal:8545": context deadline exceeded`. However, if I manually run `ufw allow 8545` then the above `geth attach` command works. I'm trying to figure out how to enable just local containers access to 8545 – Paymahn Moghadasian Mar 10 '22 at 16:16
  • Hmm, I thought ufw would use iptables under the hood. Would be interested to hear how this fits together when you figure it out! – Arnout Engelen Mar 11 '22 at 08:14