I cannot get nodes to communicate with one another in a setup with a handful of nodes. My kubernetes setup with CNI configuration is very minimal: I do kubeadm init --pod-network-cidr=192.168.0.0/16 and then I join my other two hosts.
I was trying to use calico, following the steps given in each of the on-premises deployments and quickstart. These allow the setup to complete, but there will be pods that have some failures or difficulties. After deploying calico, I see this:
NAMESPACE NAME READY STATUS RESTARTS AGE
calico-system calico-kube-controllers-79b7858c75-kddw9 0/1 Running 2 (35s ago) 115s
calico-system calico-node-b4tgx 0/1 Running 0 116s
calico-system calico-node-j9w9m 0/1 Running 0 116s
calico-system calico-node-rdf77 0/1 Running 0 116s
That I know of, the READY should be 1/1 for each of those as well. I was seeing a lot of connection is unauthorized when looking through the logs of these calico-node and calico-kube-controller.
I downloaded [flannel](curl https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml > kube-flannel.yml) and tried to modify it to specify my interfaces as an arg to flanneld. After trying all of my interfaces, flannel still isn't coming up successfully. I tried to set it to use the pod cidr that I gave in the init command, but that didn't work.
For my application, I create jobs or replica sets for my analyses, which I want to distribute over many machines. I have a service underneath which I have a job that runs a broker. Analysis pods go to a broker when they want to be given a job. It has always been the problem that these pods cannot all talk to the broker. I have configured, and re-configured, and cleansed with fire, and re-installed OS and tried other CNI solutions, which include calico and flannel. I have spent so many hours searching the internet for a fix for calico to let these pods talk to the broker.
I want a CNI solution that allows my pods to talk and works out of the box. I don't have any reason to not let all the pods talk to each other. So if there is some allow all, I want it. I am so furious after hours and hours of working with calico and flannel.
