0

I am looking for a way to decouple Prometheus from applications by putting a Kafka in between to achieve something like this:

+-------------+             +----------+       +--------------+
| Application +--metrics--->+  Kafka   +------>+ Prometheus   |
+-------------+             +----------+       +--------------+

In order to solve this problem I have two questions:

  • Are there any Java libraries that abstract metrics representation so my app will not depend on Prometheus in any ways?
  • Are there any reliable Kafka reporter?

Any comments or suggestions are welcome.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Stephen L.
  • 509
  • 2
  • 14

2 Answers2

2

The Prometheus Java client library is designed so that you can used it with other monitoring systems, and indeed many open source and commercial monitoring systems do so as the Prometheus text format is becoming a defacto-standard.

Prometheus is a pull based system and it is not at all recommended to try and convert it to push, you're making your life harder for no real reason. It is recommended to have Prometheus scrape the application directly.

brian-brazil
  • 31,678
  • 6
  • 93
  • 86
  • The diagram above shows the logical representation. The problems that I am trying to solve is to decouple Prometheus from knowing about applications as I have a zoo of them. – Stephen L. May 24 '18 at 23:01
  • You monitoring system has to know about your applications, that's the whole point really. Otherwise it can't for example tell you if they're not working. – brian-brazil May 25 '18 at 06:13
  • 1
    I disagree with this statement. In my opinion the responsibility of the app is to produce the proper metrics. The monitoring systems has to consume the metrics. There is no necessity that they should know about each other. – Stephen L. May 25 '18 at 14:42
  • 1
    Prometheus is probably not for you then. Trying to convert Prometheus to push rarely ends well. – brian-brazil May 25 '18 at 14:54
  • Is there a Kafka connector for Prometheus? – powder366 Mar 20 '19 at 10:37
0

Are there any Java libraries that abstract metrics representation so my app will not depend on Prometheus in any ways?

Yes - StatsD (I've used Datadog's dogstatsd), which is push-based.

Related - Which StatsD client should I use for a java/grails project?

Then you can use StatsD as a relay to Prometheus - https://github.com/prometheus/statsd_exporter

Or use something else that supports remote reads

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245