21

I am writing an application that is composed of a few spring boot based microservices with a zuul based reverse proxy in the front-

It works when I start the services on my machine, but for server rollout I'd like to use docker for the services, but this seems not to be possible right now.

Normally you would have a fixed "internal" port and randomized ports at the outside of the container. But the app in the container doesn't know the outside port (and IP).

The Netflix tools match what I would want to write an efficient microservice architecture and conceptually I really like docker. As far as I can see it would be very troublesome to start the container, gather the outside port on the host and pass it to the app, because you can't simply change the port after the app is started.

Is there any way to use eureka with docker based clients?

[Update] I guess I did a poor job explaining the problem. So maybe this clarifies it a bit more:

The eureka server itself can run in docker, as I have only one and the outside port doesn't matter. I can use the link feature to access it from the clients.

The problem is the URL that the clients register themselves with. This is for example https://localhost:8080/ but due to dynamic port assignment it is really only accessible via https://localhost:54321/

So eureka will return the wrong URL for the services.

UPDATE I have updated my answer below, so have a look there.

Patrick Cornelissen
  • 7,968
  • 6
  • 48
  • 70
  • You have 14 docker images with Eureka on https://registry.hub.docker.com/search?q=eureka&searchfield= and 3 zuul https://registry.hub.docker.com/search?q=zuul&searchfield= , maybe you should have a look at the associated Dockerfiles – user2915097 Apr 30 '15 at 15:36
  • The problem is not having a docker image with eureka. The problem is that the clients can't register in eureka with their randomized outside world port, as the app itself only knows the internal port. – Patrick Cornelissen Apr 30 '15 at 15:38
  • >>> the app itself only knows the internal port --> you can use a volume to write and read the internal port, see https://docs.docker.com/userguide/dockervolumes/ see `docker run --volumes-from` – user2915097 Apr 30 '15 at 15:39
  • Well, there are ways to get the external Port on the host, then there is still the task to get the eureka client use this instead of the detected internal port. I was hoping someone already tried to use this combination. – Patrick Cornelissen Apr 30 '15 at 15:50
  • do you run more docker instances (of the same service) on one machine or why can't you have fixed (outside) ports also? – sodik May 01 '15 at 21:05
  • Yes I plan to run multiple instances per machine and I plan to use one of the automated systems to start and shut down instances, so it would be good to be able to use the dynamic port stuff from docker – Patrick Cornelissen May 02 '15 at 08:18
  • This is probably a stupid question, but does Eureka really need to know the port on which it is listening externally? – larsks May 07 '15 at 12:07
  • Eureka doesn't need to know the port of itself, but the clients register with their URL at the eureka server and this url is wrong when the outside port ist not the same as the outside port. And there is the problem that the hostname/ip might be wrong, but this should be solvable by other means, as it's fairly static in contrast to the clients outside port. – Patrick Cornelissen May 07 '15 at 18:23
  • Uups, typo... "this url is wrong when the outside port ist not the same as the outside port." should read "this url is wrong when the outside port ist not the same as the inside port." (meaning the port that the spring app has inside of the docker instance). Example: The app starts with 8080 in docker and has the external port 4321, then it will register with localhost:8080 which is wrong and not accessible. A reverse proxy would rewrite this, but docker doesn't offer something like this (yet?). – Patrick Cornelissen Dec 14 '15 at 09:02
  • did you find a cleaner workaround for this (Docker still didn't seem to get around to this) – pvgoddijn Nov 11 '16 at 16:00
  • My current solution is to have a few containers with fixed ports (The reverse proxy for frontend/backend), the other services don't have exposed ports and are only accessible in the docker internal networking stack. For development I start the services via maven and infrastructure services are started with exposed ports. For this I use different docker-compose yamls which handle aspects like exposed ports. – Patrick Cornelissen Nov 12 '16 at 09:49

2 Answers2

7

I have found a solution myself, which is maybe not the best solution, but it fits for me...

When you start docker with "--net=host" (host networking), then you use the hosts network stack directly. Then I just use 0 as port for spring-boot and spring randomizes the port for me and as it's using the hosts networking stack there is no translation to a different port (and IP).

There are some drawbacks though:

  • When you use host networking you can't use the link-feature for these containers as link source or target.
  • Using the hosts network stack leads to less encapsulation of the instance, which maybe a problem depending on your project.

I hope it helps

A lot of time has passed and I think I should elaborate this a little bit further:

  1. If you use docker to host your spring application, just don't use a random port! Use a fixed port because every container gets his own IP anyway so every service can use the same port. This makes life a lot easier.

  2. If you have a public facing service then you would use a fixed port anyway.

  3. For local starts via maven or for example the command line have a dedicated profile that uses randomized ports so you don't have conflicts (but be aware that there are or have been a few bugs surrounding random ports and service registration)

  4. if for whatever reason you want to or need to use host networking you can use randomized ports of course, but most of the time you shouldn't!

Patrick Cornelissen
  • 7,968
  • 6
  • 48
  • 70
  • In AWS each service would be on it's own host, right? Then you could just use the "regular" (aka Netflix) way and use a fixed port, right? – Patrick Cornelissen Jul 03 '15 at 06:31
  • there is no problem with port, problem is with the host. Inside container I cannot determine host's IP address. However AWS SDK is able to do this and I solve my problem programmatically. – Vova Rozhkov Jul 06 '15 at 08:58
4

You can set up a directory for each docker instance and share it between the host and the instance and then write the port and IP address to a file in that directory.

$ instanceName=$(generate random instance name)
$ dirName=/var/lib/docker/metadata/$instanceName
$ mkdir -p $dirName
$ docker run -name $instanceName -v ${dirName}:/mnt/metadata ...
$ echo $(get port number and host IP) > ${dirName}/external-address

Then you just read /mnt/metadata/external-address from your application and use that information with Eureka.

Raniz
  • 10,882
  • 1
  • 32
  • 64
  • Thanks for that approach!I thought about something like that. This would require to have a special startup script that waits for the Port and IP data to be set before starting the app. As I plan to use something like kubernetes later for scaling and management I want to keep the need for special scripts as low as possible. – Patrick Cornelissen May 08 '15 at 05:55
  • You can just poll for the file in your app and register with Eureka once it is populated. Though it does require special handling when starting containers. – Raniz May 08 '15 at 07:06
  • Correct, as long as host networking mode is no problem I'll use that. When I have to go to standard networking, I'll try your suggestion. Thanks! – Patrick Cornelissen May 08 '15 at 07:24