24

I want to calculate a ratio of two metrics, but I get no data...

I have some metrics like:

fs_bytes{filesystem="/var",instance="localhost:9108",job="graphite",metric="Used"}   50.0
fs_bytes{filesystem="/var",instance="localhost:9108",job="graphite",metric="Total"}   100.0

When I try to do any operation (device, multiply, add, subtract) like:

fs_bytes{instance="localhost:9108",metric="Used"} / fs_bytes{instance="localhost:9108",metric="Total"}

Prometheus returned:

no data

When I query each metric alone in the Prometheus expression browser, I do get the metrics values.

What's wrong?

Adrian Mole
  • 49,934
  • 160
  • 51
  • 83
Franklin Piat
  • 3,952
  • 3
  • 32
  • 45

1 Answers1

34

When prometheus is evaluating an expression, the operation implicitly apply to metric that share identical set of labels.

Despite the fact I specified the metric name and most labels, Prometheus was looking for metrics that have the same set of labels.

However, in this case, two metrics have different label values, so they can't match ! (one metric has metric="Used"the other has metric="Total". It could be that one of the metrics has some extra labels).

The solution is to use ignore (or on) to reduce the set of considered labels:

fs_bytes{instance="localhost:9108",metric="Used"} / ignoring(metric) fs_bytes{instance="localhost:9108",metric="Total"}

Read the fine manual ! (here)

Franklin Piat
  • 3,952
  • 3
  • 32
  • 45
  • The solution is also explained in the documentation https://prometheus.io/docs/prometheus/latest/querying/operators/#one-to-one-vector-matches although not as clearly as in this answer. – Juuso Ohtonen May 11 '21 at 08:24