2

I'm trying to migrate applications based on the Netflix OSS to Kubernetes so the ideal way I found was to create a service of type NodePort and register the applications to Eureka. So i'm doing eureka.hostname=hostIP and eureka.nonSecurePort=nodePort

Here's what I've done -

  1. Create a service for sample-app-service with service type NodePort.

  2. Inject the nodeport in to a ConfigMap by running the command kubectl create configmap saas-event-reception-config --from-literal=nodePort=$(kubectl get -o jsonpath="{.spec.ports[0].nodePort}" services sample-app-service) (Question: Is there a way I can specify this as a yaml?)

  3. Refer the nodePort using the configMapKeyRef in the deployment yaml.

The problem I'm facing is during the automated deployment. So ideally I'd like deploy the application using a single deployment file which includes Service, ConfigMap and Deployment. Is there a way I can do this gracefully? Or are there any alternate suggestion for doing this.

I'm also looking at helm but even if I use --set to pass the nodePort to ConfigMap by running the command (kubectl get -o jsonpath="{.spec.ports[0].nodePort}" services sample-app-service) the Service has to be deployed first so that the ConfigMap gets the nodePort value. Is there a way I can do this?

Community
  • 1
  • 1
Zuke
  • 31
  • 4
  • Not sure why you're not using the internal service names (https://stackoverflow.com/a/54051377/9705485) rather than the external nodePorts? Is your eureka not running in k8s? – Ryan Dawson Jan 14 '19 at 14:07
  • @RyanDawson - The eureka is on k8s but we have a multi data center setup where each k8s cluster has a eureka and there's a peer replication so each app registers to both the eureka services so that's where I'm looking at using the nodePorts. – Zuke Jan 14 '19 at 14:38
  • @RyanDawson - Quick question - How does eureka resolve the k8s service name? How does it work in case of app replicas that register to eureka? – Zuke Jan 14 '19 at 15:13
  • Basically eureka just gets a name registered and multiple instances registering the same name is fine. But if you've got multiple eurekas in different clusters then I can see how the situation is more complex for you. If you want to be able to make cross-cluster calls then you would have to use the external names. Presumably you are sharing traffic between different clusters in case one cluster goes down? – Ryan Dawson Jan 14 '19 at 15:43
  • That exactly the case. If the nodes in one cluster go down the traffic can still be redirected to the nodes in the other cluster. – Zuke Jan 14 '19 at 17:37
  • So the thing you're looking to improve is your deployment process having that manual step where you find out the nodePort and put it into the configmap. If you know your nodeport range (default is 30000-32767) then you can specify which nodePort to use in your yaml files https://github.com/ryandawsonuk/minions/blob/4d564ea2486c3d39cbada43e2cfec17e5db69881/minion-army.yml#L45 so that you'll know it in advance – Ryan Dawson Jan 14 '19 at 18:05

1 Answers1

0

What you can do is to specify a port from the service-node-port-range (by default 30000-32767) in your Service yaml file under ports with the name nodePort. Then you'll know the nodePort in advance. You could use helm to pass in the nodePort to use as a parameter so that this parameter is used in the Service yaml and also your ConfigMap and Deployment.

Ryan Dawson
  • 11,832
  • 5
  • 38
  • 61
  • Another way to do it would be to first install and expose an Ingress controller and use ingress-based URLs, which you could also know in advance. But you already have NodePort working so easiest to stick with it. – Ryan Dawson Jan 15 '19 at 08:15
  • Thanks @RyanDawson for your answers. The problem with this approach is I do not want to specify nodePorts expicitly because I'll run into problems eventually when the number of microservices increase and nodePort will start conflicting. – Zuke Jan 16 '19 at 07:17
  • Another other way is to set up an ingress controller and expose that via NodePort. It is common to treat the ingress controller as a piece of infrastructure rather than part of your app deployment. Then you can specify ingress resources for each service and use the ingress URLs for each microservice. I am not sure whether eureka will accept URLs distinguished by path or if you'll have to put them on different external hosts by setting up wildcard DNS. You'd have to try it. – Ryan Dawson Jan 16 '19 at 08:08
  • I'm assuming you are running on-prem and not using a managed service and that's why you are using NodePort and running multiple clusters. I think you only need a different NodePort per distinct microservice type and not per instance/replica. Given the size of the NodePort range, are you really going to run into conflicts on that? – Ryan Dawson Jan 16 '19 at 08:17
  • Another thought - presumably your eurekas are hooked up to Zuul instances. Could you just use the internal names when registering microservices with eurekas and only expose Zuul externally and put a load-balancer in front so that you load-balance between the Zuuls in the different clusters. I'm assuming your existing app is very committed to the Netflix components as otherwise it may be easier to remove them and go for a pure k8s approach. – Ryan Dawson Jan 16 '19 at 08:24
  • Also think about whether you really need multiple clusters and not a single high availability cluster. – Ryan Dawson Jan 16 '19 at 08:45