how to create custom gauges with Spring Boot and Micrometer?

Question

I wrote a Spring Boot app and I'd like to expose custom metrics to Prometheus with Micrometer. Here's a code snippet where I'm incrementing a counter (which works) and attempts to set a couple of gauges (which doesn't work):

@Scheduled(cron = "*/2 * * * * *") // run every 2 seconds
private void logTemperature() throws UnknownHostException {

    // get measurement
    SensorReader sensorReader = new SensorReader();
    SensorReading sensorReading = sensorReader.getSensorReading();

    Metrics.counter("measurements").increment();
    Metrics.gauge("fahrenheit", sensorReading.getFahrenheit());
    Metrics.gauge("humidity", sensorReading.getHumidity());

    // do stuff with measurement
    // ...

}

The complete source code is here: https://github.com/alexwoolford/htu21d_logger/tree/master/htu21d-logger-spring

The fahrenheit field in the sensorReading object is a float. The gauges exist in the output, however the value is NaN (i.e. not a number):

# HELP measurements_total  
# TYPE measurements_total counter
measurements_total 82.0
...
# HELP humidity  
# TYPE humidity gauge
humidity NaN
...
# HELP fahrenheit  
# TYPE fahrenheit gauge
fahrenheit NaN
...

I've seen similar posts, e.g. Micrometer/Prometheus How do I keep a gauge value from becoming NaN? and Micrometer - Prometheus Gauge displays NaN, but I can't figure out what, specifically, I need to do from those answers.

I also tried creating a variable and updating that with a lambda, e.g.

Metrics.gauge("humidity", humidity, humidity -> sensorReading.getHumidity());

... and the guage disappeared altogether (not even an NaN).

This question is slightly different from the others since I'm not using a cache and I'm attempting to use the global registry which, presumably, is the simplest option.

score 10 · Accepted Answer · answered Jul 31 '19 at 15:34

The objects that you are using are getting garbage collected since noting is holding onto a reference of those values. Gauges use weak references by default.

If you set it to use strong references it should avoid the NaNs. Unfortunately the global registry helpers don't expose the gauge builder so you'll need to create the gauge a little differently.

Gauge.builder("humidity", humidity, humidity -> sensorReading.getHumidity()).strongReference(true).register(Metrics.globalRegistry);

As a side note I would recommend moving away from the @Scheduled approach unless the sensor reading is inherently slow. If the SensorReader is thread safe, you could remove the @Scheduled and use the SensorReader directly:

SensorReader sensorReader = new SensorReader();

Gauge.builder("humidity", sensorReader, sensorReader -> 
  sensorReader.getSensorReading().getHumidity())
  .strongReference(true)
  .register(Metrics.globalRegistry);
Gauge.builder("fahrenheit", sensorReader, sensorReader -> 
  sensorReader.getSensorReading().getFahrenheit())
  .strongReference(true)
  .register(Metrics.globalRegistry);

That way you'll only be looking up the sensor reading are the rate that Prometheus is scraping the metrics.

That's awesome, @checketts. Thank you so much. – Alex Woolford Jul 31 '19 at 21:35 — Alex Woolford, Jul 31 '19 at 21:35

how to create custom gauges with Spring Boot and Micrometer?

1 Answers1