How to enable containers internet access through flannel?

Question

Problem:

/ # ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
ping: sendto: Network unreachable

Example container ifconfig:

eth0      Link encap:Ethernet  HWaddr F2:3D:87:30:39:B8
      inet addr:10.2.8.64  Bcast:0.0.0.0  Mask:255.255.255.0
      inet6 addr: fe80::f03d:87ff:fe30:39b8%32750/64 Scope:Link
      UP BROADCAST RUNNING MULTICAST  MTU:1450  Metric:1
      RX packets:22 errors:0 dropped:0 overruns:0 frame:0
      TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:0
      RX bytes:4088 (3.9 KiB)  TX bytes:648 (648.0 B)

eth1      Link encap:Ethernet  HWaddr 6E:1C:69:85:21:96
      inet addr:172.16.28.63  Bcast:0.0.0.0  Mask:255.255.255.0
      inet6 addr: fe80::6c1c:69ff:fe85:2196%32750/64 Scope:Link
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:18 errors:0 dropped:0 overruns:0 frame:0
      TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:0
      RX bytes:1418 (1.3 KiB)  TX bytes:648 (648.0 B)

lo        Link encap:Local Loopback
      inet addr:127.0.0.1  Mask:255.0.0.0
      inet6 addr: ::1%32750/128 Scope:Host
      UP LOOPBACK RUNNING  MTU:65536  Metric:1
      RX packets:0 errors:0 dropped:0 overruns:0 frame:0
      TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:0
      RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

Routing inside container:

/ # ip route show
10.2.0.0/16 via 10.2.8.1 dev eth0
10.2.8.0/24 dev eth0  src 10.2.8.73
172.16.28.0/24 via 172.16.28.1 dev eth1  src 172.16.28.72
172.16.28.1 dev eth1  src 172.16.28.72

Host iptables: http://pastebin.com/raw/UcLQQa4J

Host ifconfig: http://pastebin.com/raw/uxsM1bx6

logs by flannel:

main.go:275] Installing signal handlers
main.go:188] Using 104.238.xxx.xxx as external interface
main.go:189] Using 104.238.xxx.xxx as external endpoint
etcd.go:129] Found lease (10.2.8.0/24) for current IP (104.238.xxx.xxx), reusing
etcd.go:84] Subnet lease acquired: 10.2.8.0/24
ipmasq.go:50] Adding iptables rule: FLANNEL -d 10.2.0.0/16 -j ACCEPT
ipmasq.go:50] Adding iptables rule: FLANNEL ! -d 224.0.0.0/4 -j MASQUERADE
ipmasq.go:50] Adding iptables rule: POSTROUTING -s 10.2.0.0/16 -j FLANNEL
ipmasq.go:50] Adding iptables rule: POSTROUTING ! -s 10.2.0.0/16 -d 10.2.0.0/16 -j MASQUERADE
vxlan.go:153] Watching for L3 misses
vxlan.go:159] Watching for new subnet leases
vxlan.go:273] Handling initial subnet events
device.go:159] calling GetL2List() dev.link.Index: 3
vxlan.go:280] fdb already populated with: 104.238.xxx.xxx 82:83:be:17:3e:d6
vxlan.go:280] fdb already populated with: 104.238.xxx.xxx 82:dd:90:b2:42:87
vxlan.go:280] fdb already populated with: 104.238.xxx.xxx de:e8:be:28:cf:7a
systemd[1]: Started Network fabric for containers.

Your container does not have default route defined, so it's not going to be able to reach anything other than directly connected networks. — larsks, May 07 '16 at 00:00
I suspected as much, the thing is that routing is supposed to be configured in the `ipam` field of the rkt network declaration, but the flannel plugin manages the `ipam` field, you can change other settings through the `delegate` field but not the `ipam` so I thought flannel was supposed to handle that?! — untorhe, May 07 '16 at 11:27

score 2 · Answer 1 · edited Jun 22 '18 at 16:26

2

It is possible if you set a config map with upstreamNameServers.

Example:

apiVersion: v1
kind: ConfigMap
metadata:
  name: kube-dns
  namespace: kube-system
data:
  upstreamNameservers: |
    ["8.8.8.8", "8.8.8.4"]

And in you Deployment definition add:

dnsPolicy: "ClusterFirst"

More info here:

https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers

edited Jun 22 '18 at 16:26

VitorMM

1,060
8
33

answered Jun 22 '18 at 14:43

Bruck Wubete

141
2
5

untorhe · Accepted Answer · 2016-05-08T20:27:55.707

1

It is not possible to make it work because it is not yet implemented...I guess I am switching to docker...

edit: ...or not, switched from flannel to calico, it works ok.

rkt #862
k8s #2249

edited May 08 '16 at 20:27

answered May 07 '16 at 13:26

untorhe

31
1
5

score 1 · Answer 3 · answered Dec 02 '16 at 02:11

This GitHub issue on the Flannel project may provide a solution - essentially, try disabling IP masquerading (--ip-masq=false) on your Docker daemon, and enabling it (--ip-masq) on your Flannel daemon.

This solution worked for me when I was unable to ping internet IPs (e.g. 8.8.8.8) from inside a container in my Kubernetes cluster.

score 0 · Answer 4 · answered Sep 18 '20 at 07:51

Try to Check the Kube-flannel.yml file and also the starting command to create the cluster that is kubeadm init --pod-network-cidr=10.244.0.0/16 and by default in this file kube-flannel.yml you will get the 10.244.0.0/16 IP, so if you want to change the pod-network-CIDR then please change in the file also.

How to enable containers internet access through flannel?

4 Answers4