2

I’m having a bit of trouble getting Turbine to work in Spring Cloud. In a nutshell, I can’t determine how to configure it to aggregate circuits from more than one application at a time.

I have 6 separate services, a eureka server, and a turbine server running in standalone mode. I can see from my Eureka server that all of the services are registered, including turbine. My turbine server is up and running, and I can see its /hystrix page without issue. But when I try to use it to examine turbine.stream, I only see the FIRST server that is listed in turbine.appConfig, the rest are ignored.

This is my Turbine server’s application.yml, or at least the relevant parts:

---
eureka:
  client:
    serviceUrl:
      defaultZone: http://localhost:8010/eureka/
server:
  port: 8030
info:
  component: Turbine
turbine:
  clusterNameExpression: new String(“default”)
  appConfig: sentence,subject,verb,article,adjective,noun
management:
  port: 8990

When I run this and access the hystrix dashboard on my turbine instance, asking for the turbine.stream, the ONLY circuit breakers listed in the output are for the first service listed in appConfig, the “sentence” service in this case. Curiously, if I re-arrange the order of these services and put another one first (like “noun”), I see only the circuits for THAT service. Only the first service in the list is displayed.

I’ll admit to being a little confused on some of the terminology, like streams, clusters, etc., so I could be missing some basic concept here, but my understanding is that Turbine could digest streams from more than one service and aggregate them in a single display. Suggestions would be appreciated.

Ken Krueger
  • 1,005
  • 14
  • 26
  • what url are you using in the hystrix dashboard? – spencergibb Apr 30 '15 at 16:00
  • Hi @spencergibb, thanks for the response. I use http://localhost:8030/hystrix to access turbine, and http://localhost:8030/turbine.stream within the hystrix dashboard. I've tried my machine name instead of localhost as well, same result. – Ken Krueger May 01 '15 at 17:22
  • You need to put `/turbine.stream?cluster=` – spencergibb May 01 '15 at 17:32
  • Thanks Spencer. What value does one use for the cluster name? I was attempting to use the default cluster. Whether I use /turbine.stream or /turbine.stream?cluster=default I get the same result, only the FIRST client listed in appConfig appears in the dashboard. How do I get all of them to appear? – Ken Krueger May 03 '15 at 17:47
  • Please see the documentation: http://projects.spring.io/spring-cloud/docs/1.0.1/spring-cloud.html#_turbine In your case, `turbine.aggregator.clusterConfig=SENTENCE,SUBJECT,VERB,ARTICLE,ADJECTIVE,NOUN` and `` would be one of those values. I would not use `clusterNameExpression`. – spencergibb May 04 '15 at 15:08
  • Thanks for your help Spencer. I’ve been re-reading that section of the reference quite a bit over the last few days. For clarification: I’m trying to use turbine for a consolidated monitor of multiple services, not a single service. By your suggestion, I’ve hard-configured all service IDs for turbine.aggregator.clusterConfig, and repeating this in turbine.appConfig properties. And this does allow me to successfully obtain output for one specific cluster at a time. But I’m seeking a consolidated dashboard of all services, preferably without any hard-coding. – Ken Krueger May 06 '15 at 20:08
  • Also, regarding avoiding clusterNameExpression, can you explain why? It appears to offer exactly what I need. The thread at http://stackoverflow.com/questions/28365202/spring-boot-eureka-server-hystrix-with-turbine-empty-turbine-stream indicates that this should actually be clusterNameExpression: new String(“default”). I’ve verified by debugging the code in o.s.c.n.turbine.EurekaInstanceDiscovery that this does indeed assign all of the instances to the “default” cluster. But even still only one app’s breakers appear on the monitor. – Ken Krueger May 06 '15 at 20:09

1 Answers1

6

I don't have enough reputation to comment, so I have to write this in an answer :)

I had the exactly same problem:

There are two services "test-service" and "other-service", each with it's own working hystrix-stream and there is one Turbine-Application, which is configured like this:

    turbine:
        clusterNameExpression: new String("default")
        appConfig: test-service,other-service

All of my services are running on my local machine.

Result is: My Hystrix-Dashboard just shows the metrics from "test-service".

Reason:

It seems to be, that a Turbine-Client which is configured the described way doesn't handle multiple services when they are running at the same host.

This is explained here:

https://github.com/Netflix/Hystrix/issues/117#issuecomment-14262713

Turbine maintains state of all these instances in order to maintain persistent connections to them and it does rely on the "hostname" and if the host name is the same then it won't instantiate a new connection to that same server (on a different port).

So the main point is, that all of your services must be registered with different hostnames. How you could do this on your local machine is described below.

UPDATE 2015-06-12/2016-01-23: Workaround for local testing

Change your hostfile:

    # ...
    127.0.0.1 localhost
    127.0.0.1 localdomain1
    127.0.0.1 localdomain2
    # ...
    127.0.0.1 localdomainx

And then set the hostname for your clients each to a different domain-entry like this:
application.yml: 

    eureka:
        instance:
            hostname: localdomainx
Mike Boddin
  • 1,241
  • 2
  • 12
  • 22
  • Bingo! The hostname differentiates the multiple services. I think you should get more experience points Mike, but I'm glad you posted this as an answer. The only other thing I noticed is that even if you have different host names, if the method with the circuit breaker has the same name, they are aggregated together. – Ken Krueger Jun 15 '15 at 18:48