15

I can't connect to my app running with nginx ingress (Docker Desktop win 10).

The nginx-ingress controller pod is running, the app is healthy, and I have created an ingress. However, when I try to connect to my app on localhost, I get "connection refused".

I see this error in the log:

[14:13:13.028][VpnKit            ][Info   ] vpnkit.exe: Connected Ethernet interface f6:16:36:bc:f9:c6

[14:13:13.028][VpnKit            ][Info   ] vpnkit.exe: UDP interface connected on 10.96.181.150

[14:13:22.320][GoBackendProcess  ][Info   ] Adding vpnkit-k8s-controller tcp forward from 0.0.0.0:80 to 10.96.47.183:80
[14:13:22.323][ApiProxy          ][Error  ] time="2019-12-09T14:13:22-05:00" msg="Port 443 for service ingress-nginx is already opened by another service"

I think port 443 is used by another app, possibly zscaler security or skype. Excerpt from netstat -a -b:

 [svchost.exe]
  TCP    0.0.0.0:443            0.0.0.0:0              LISTENING       16012
 [com.docker.backend.exe]
  TCP    0.0.0.0:443            0.0.0.0:0              LISTENING       8220

I don't know how to make the ingress work. Please help!

My ingress:

$ kubectl describe ing kbvalues-deployment-dev-ingress
Name:             kbvalues-deployment-dev-ingress
Namespace:        default
Address:          localhost
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host       Path  Backends
  ----       ----  --------
  localhost
             /   kbvalues-deployment-dev-frontend:28000 (10.1.0.174:8080)
Annotations:
  kubernetes.io/ingress.class:                     nginx
  nginx.ingress.kubernetes.io/cors-allow-headers:  X-Forwarded-For, X-app123-XPTO
Events:
  Type    Reason  Age               From                      Message
  ----    ------  ----              ----                      -------
  Normal  CREATE  42m               nginx-ingress-controller  Ingress default/kbvalues-deployment-dev-ingress
  Normal  UPDATE  6s (x5 over 42m)  nginx-ingress-controller  Ingress default/kbvalues-deployment-dev-ingress

My service:

$ kubectl describe svc kbvalues-deployment-dev-frontend
Name:              kbvalues-deployment-dev-frontend
Namespace:         default
Labels:            chart=tomcat-sidecar-war-1.0.4
                   environment=dev
                   name=kbvalues-frontend-dev
                   release=kbvalues-test
                   tier=frontend
Annotations:       <none>
Selector:          app=kbvalues-dev
Type:              ClusterIP
IP:                10.98.89.94
Port:              <unset>  28000/TCP
TargetPort:        8080/TCP
Endpoints:         10.1.0.174:8080
Session Affinity:  None
Events:            <none>

I am trying to access the app at: http://localhost:28000/health. I verified that the /health URL is accessible locally within the web server container.

I appreciate any help you can offer.

Edit:

I tried altering the ingress-nginx service to remove HTTPS, as suggested here: https://stackoverflow.com/a/56303330/166850

This got rid of the 443 error in the logs, but didn't fix my setup (still getting connection refused).

Edit 2: Here is the Ingress YAML definition (kubectl get -o yaml):

$ kubectl get ing -o yaml
apiVersion: v1
items:
- apiVersion: extensions/v1beta1
  kind: Ingress
  metadata:
    annotations:
      kubernetes.io/ingress.class: nginx
    creationTimestamp: "2019-12-09T18:47:33Z"
    generation: 5
    name: kbvalues-deployment-dev-ingress
    namespace: default
    resourceVersion: "20414"
    selfLink: /apis/extensions/v1beta1/namespaces/default/ingresses/kbvalues-deployment-dev-ingress
    uid: 5c34bf7f-1ab4-11ea-80e4-00155d169409
  spec:
    rules:
    - host: localhost
      http:
        paths:
        - backend:
            serviceName: kbvalues-deployment-dev-frontend
            servicePort: 28000
          path: /
  status:
    loadBalancer:
      ingress:
      - hostname: localhost
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

Edit 3: Output of kubectl get svc -A (ingress line only):

ingress-nginx   ingress-nginx                      LoadBalancer   10.96.47.183     localhost     80:30470/TCP             21h

Edit 4: I tried to get the VM's IP address from windows HyperV, but it seems like the VM doesn't have an IP?

PS C:\> (Get-VMNetworkAdapter -VMName DockerDesktopVM)

Name            IsManagementOs VMName          SwitchName MacAddress   Status IPAddresses
----            -------------- ------          ---------- ----------   ------ -----------
Network Adapter False          DockerDesktopVM DockerNAT  00155D169409 {Ok}   {}

Edit 5:

Output of netstat -a -n -o -b for port 80:

  TCP    0.0.0.0:80             0.0.0.0:0              LISTENING       4
 Can not obtain ownership information
RMorrisey
  • 7,637
  • 9
  • 53
  • 71
  • 1
    Are you able to access the service without Ingress? Like directly hitting the service IP and port works? – Shubham Dec 09 '19 at 19:59
  • 1
    You should try doing `kubectl exec -it /bin/bash`, run the curl command for what Shubham suggested and check your outputs. – randominstanceOfLivingThing Dec 09 '19 at 20:05
  • 1
    What's the output of `kubectl get svc -A` (only pasting the ingress line)? I suspect your ingress service is not configured correctly. For cloud environments, i'd expect the Type to be `LoadBalancer` however for your case, I think you may need to configure it to use `NodePort` where you'd access your ingress using a port (E.g., https://localhost:30252/myapp). – leeman24 Dec 09 '19 at 20:18
  • 1
    @Shubham @randominstanceOfLivingThing I was able to access the service successfully using: ```kubectl port-forward svc/kbvalues-deployment-dev-frontend 28000``` and hitting: ```http://localhost:28000/health``` – RMorrisey Dec 10 '19 at 14:11
  • @RMorrisey if your comment solved your problem please provide an answer to your post and accept it for future reference. – Dawid Kruk Dec 10 '19 at 14:50
  • @DawidKruk Sorry for the confusion - my problem is still unsolved. The above command is only a workaround; it doesn't fix the ingress functionality as described in the OP. – RMorrisey Dec 10 '19 at 15:00
  • Does this answer your question? [Enable Ingress controller on Docker Desktop with WLS2](https://stackoverflow.com/questions/65193758/enable-ingress-controller-on-docker-desktop-with-wls2) – Michael Freidgeim Mar 06 '21 at 06:32

5 Answers5

30

I have managed to create Ingress resource in Kubernetes on Docker in Windows.

Steps to reproduce:

  • Enable Hyper-V
  • Install Docker for Windows and enable Kubernetes
  • Connect kubectl
  • Enable Ingress
  • Create deployment
  • Create service
  • Create ingress resource
  • Add host into local hosts file
  • Test

Enable Hyper-V

From Powershell with administrator access run below command:

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V -All

System could ask you to reboot your machine.

Install Docker for Windows and enable Kubernetes

Install Docker application with all the default options and enable Kubernetes

Connect kubectl

Install kubectl .

Enable Ingress

Run this commands:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/cloud-generic.yaml

Edit: Make sure no other service is using port 80

Restart your machine. From a cmd prompt running as admin, do: net stop http Stop the listed services using services.msc

Use: netstat -a -n -o -b and check for other processes listening on port 80.

Create deployment

Below is simple deployment with pods that will reply to requests:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello
spec:
  selector:
    matchLabels:
      app: hello
      version: 2.0.0
  replicas: 3
  template:
    metadata:
      labels:
        app: hello
        version: 2.0.0
    spec:
      containers:
      - name: hello
        image: "gcr.io/google-samples/hello-app:2.0"
        env:
        - name: "PORT"
          value: "50001"

Apply it by running command:

$ kubectl apply -f file_name.yaml

Create service

For pods to be able for you to communicate with them you need to create a service.

Example below:

apiVersion: v1
kind: Service
metadata:
  name: hello-service
spec:
  type: NodePort
  selector:
    app: hello
    version: 2.0.0
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 50001

Apply this service definition by running command:

$ kubectl apply -f file_name.yaml

Create Ingress resource

Below is simple Ingress resource using service created above:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: hello-ingress 
spec:
  rules:
  - host: hello-test.internal
    http:
      paths:
      - path: /
        backend:
          serviceName: hello-service 
          servicePort: http

Take a look at:

spec:
  rules:
  - host: hello-test.internal

hello-test.internal will be used as the hostname to connect to your pods.

Apply your Ingress resource by invoking command:

$ kubectl apply -f file_name.yaml

Add host into local hosts file

I found this Github link that will allow you to connect to your Ingress resource by hostname.

To achieve that add a line 127.0.0.1 hello-test.internal to your C:\Windows\System32\drivers\etc\hosts file and save it. You will need Administrator privileges to do that.

Edit: The newest version of Docker Desktop for Windows already adds a hosts file entry: 127.0.0.1 kubernetes.docker.internal

Test

Display the information about Ingress resources by invoking command: kubectl get ingress

It should show:

NAME            HOSTS                 ADDRESS     PORTS   AGE
hello-ingress   hello-test.internal   localhost   80      6m2s

Now you can access your Ingress resource by opening your web browser and typing

http://kubernetes.docker.internal/

The browser should output:

Hello, world!
Version: 2.0.0
Hostname: hello-84d554cbdf-2lr76

Hostname: hello-84d554cbdf-2lr76 is the name of the pod that replied.

If this solution is not working please check connections with the command: netstat -a -n -o (with Administrator privileges) if something is not using port 80.

S.Sachith
  • 536
  • 1
  • 9
  • 21
Dawid Kruk
  • 8,982
  • 2
  • 22
  • 45
  • 1
    Thanks for the detailed writeup! I will try following these steps tomorrow morning and report back – RMorrisey Dec 10 '19 at 21:30
  • Sorry! I got used to working around it with kubectl proxy and forgot! I'll try it right now. – RMorrisey Dec 18 '19 at 19:33
  • Following the steps above broke my kubectl context. It now says: ``` kubectl get ing Unable to connect to the server: dial tcp: lookup kubernetes.docker.internal: no such host``` – RMorrisey Dec 18 '19 at 19:39
  • Let me find where kubernetes.docker.internal is set – RMorrisey Dec 18 '19 at 19:39
  • I changed the cluster settings in my .kube config file, but now I get: ```Unable to connect to the server: x509: certificate is valid for docker-for-desktop, kubernetes, kubernetes.default, kubernetes.default.svc, kubernetes.default.svc.cluster.local, kubernetes.docker.internal, vm.docker.internal, localhost, not hello-test.internal ``` – RMorrisey Dec 18 '19 at 19:40
  • Trying to change everything from hello-test.internal to kubernetes.docker.internal – RMorrisey Dec 18 '19 at 19:41
  • I tried the following changes: -Changed cluster config back to how it was -Changed hosts to say: 127.0.0.1 kubernetes.docker.internal -Changed ingress rule to: - host: kubernetes.docker.internal – RMorrisey Dec 18 '19 at 19:43
  • Now I am able to use kubectl once again, but I got a 404 error page from the web server accessing: ```http://kubernetes.docker.internal/``` – RMorrisey Dec 18 '19 at 19:44
  • ```kubectl get ing``` shows: hello-ingress kubernetes.docker.internal localhost 80 5m14s – RMorrisey Dec 18 '19 at 19:45
  • I scaled back the deployment to one pod for debugging, retried my test request, and checked the logs. There is no record of my HTTP request: 2019/12/18 19:36:17 Server listening on port 50001 – RMorrisey Dec 18 '19 at 19:47
  • To be clear, all of the steps above worked up until the hosts file step - that's what broke my kubectl – RMorrisey Dec 18 '19 at 19:49
  • The hosts file had an entry: ```127.0.0.1 kubernetes.docker.internal``` I changed this entry to hello-test.internal and that broke it. – RMorrisey Dec 18 '19 at 19:52
  • I tried a couple different variations (having both entries: 127.0.0.1 kubernetes.docker.internal 127.0.0.1 hello-test.internal) (having both items in the same line: 127.0.0.1 kubernetes.docker.internal hello-test.internal). In both fo these configs, kubectl was working but I still got the 404. – RMorrisey Dec 18 '19 at 19:56
  • Netstat -a -n -o -b says "Can not obtain ownership information" for port 80 – RMorrisey Dec 18 '19 at 20:00
  • The 404 page has the following label: "openresty/1.15.8.2" – RMorrisey Dec 18 '19 at 20:08
  • Restarted my PC, and stopped several services using `net stop http`. The HTTP service will not stop, but I got to the point where I get "connection refused" when accessing localhost:80. Started up docker desktop and started kubernetes local cluster. Now I am back to the 404 page (from openresty). – RMorrisey Dec 18 '19 at 20:49
  • I got it working! Thanks so much! The solution was: – RMorrisey Dec 18 '19 at 21:13
  • 1) Restart my PC and stop all services using HTTP (use 'net stop http' to find out which ones) – RMorrisey Dec 18 '19 at 21:14
  • 2) Start up docker desktop & kubernetes – RMorrisey Dec 18 '19 at 21:14
  • 3) Change hosts file and ingress to use kubernetes.docker.internal – RMorrisey Dec 18 '19 at 21:15
  • 4) Fix sample service & ingress given above so that they map to port 80 (ingress -> servicePort: http) (service -> ports: - name: http protocol: TCP port: 80 targetPort: 50001) – RMorrisey Dec 18 '19 at 21:16
  • i was facing similar issue while deploying ingress-nginx controller using the manual steps mentioned for bareMetal node at (https://kubernetes.github.io/ingress-nginx/deploy/) however was facing an issue , however referred to the link (https://github.com/docker/for-win/issues/1901) mentioned by RMorrisey which mentioned to install ingress-nginx using the steps mentioned for mac and it worked without making cny changes to host file , etc – Mayur Sep 18 '20 at 07:54
  • Thank you, I spent hours to figure it out what the problem was, all I had to do is use http://kubernetes.docker.internal/ URL, that you wrote in the last section. This should be in the docs on this page https://kubernetes.github.io/ingress-nginx/deploy/ but there is no Docker Desktop for Windows section there :( – zsolt Nov 30 '20 at 20:38
  • Little bit of update to my previous comment. I "reinstalled" all k8s related stuff to write down step-by-step what needs to be done to run my pods. So to use the http://kubernetes.docker.internal URL in Docker Desktop for WINDOWS you need to install this ingress controller too: https://kubernetes.github.io/ingress-nginx/deploy/#docker-for-mac Yes, you need that, the Docker for Mac version, it works perfectly today on Windows, Docker Desktop 2.5.0.1 (49550) – zsolt Dec 01 '20 at 15:11
  • I am using win 10 home where hyper v is not there. So what should I do ? – yogihosting Feb 19 '21 at 09:13
  • @yogihosting I have no access to a Windows 10 Home machine but I'd reckon you can use Docker Desktop with `wsl2` backend. Also you can look for other solutions like `minikube` with `virtualbox` driver. The setup for accessing ingress on your localhost machine will be slightly different but it will be also possible. – Dawid Kruk Feb 19 '21 at 09:23
  • TLDR: add ingress class! I kept getting a 404 from NGINX. I looked into the ingress controller pod logs using `kubectl logs -n ingress-nginx pod/ingress-nginx-controller-***pod-hash-here***` and found this error message: `Ignoring ingress because of error while validating ingress class" ingress="mongo-ns/mongo-ingress" error="ingress does not contain a valid IngressClass`. Turns out, I needed to add this under metadata/annotations: `kubernetes.io/ingress.class: nginx` – Igor Akkerman Aug 31 '21 at 21:45
2

On Windows the Kubernetes cluster is running in a VM. Try to access ingress on that VM-s IP address instead of localhost.

Dávid Molnár
  • 10,673
  • 7
  • 30
  • 55
1

If you want to access the ingress on localhost without any special host names, it's enough if you enable the ingress nginx controller:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml

You don't even need to specify the -host argument in your Ingress. If you have a special host name, such as my-test.com, you have to add the domain the the /etc/hosts file as the others mentioned.

I hope it helps somebody!

kirshiyin
  • 141
  • 1
  • 9
0

i was facing similar issue while deploying ingress-nginx controller using the manual steps mentioned for bareMetal node at ingress-nginx-deploy however was facing an issue , however referred to the link Github link mentioned by @RMorrisey which leads to other threads where they have mentioned to install ingress-nginx using the steps mentioned for mac and it worked without making cny changes to host file , etc

Mayur
  • 303
  • 1
  • 3
  • 14
0

The problem is that your service has a type of ClusterIP, which isn't accessible externally. You need it to be of type NodePort, which is what is done in Dawid Kruk's instructions.

Patrick Farry
  • 101
  • 1
  • 5