3

I want to create an alert that triggers whenever one of the following counter statistics is not zero:

a.b.c.failed
a.b.e.failed

I already use these statistics separately on a dashboard page, but as they occur rarely, I'd like an alert.

It appears I have to make a sum composite so that I can trigger the alert when the sum is above zero. I think the composite would look something like:

sum(series("a.b.*.failed",{}))

However, every attempt I make gives the error: 

Unable to execute composite: ["error": "Requested MD data from SD endpoint"]

There is another thread that suggested replacing the {} with "*" (including the quotes). This no longer gives an error, but gives a bizarre result (it's above zero all the time, even though there only very rarely any 'failed' statistics above zero).

Neil B.
  • 61
  • 5

1 Answers1

3

The correct expression for my case is:

sum(derive(series("a.b.*.failed","*")))

Using "*" works to select the source.

Derive gives the change of each statistic instead of the cumulative total (but I'm not sure why the cumulative total was showing up - it is not shown normally for these statistics).

Sum adds the change of the different statistics.

I don't understand why {} doesn't work - I think that is related to the mystery of the meaning the error message that uses undocumented terminology (MD and SD endpoints). Librato documentation of their composite statistics function language is very minimal and provides few examples and few explanations of the meaning of terms and technical foundations.

Neil B.
  • 61
  • 5
  • 1
    MD refers to the tag-based model of their metrics, whereas SD is the older source-based model. Using {} implies your composite is using tags (or MD), which your account may not support. Here's a link to the older documentation for reference. http://kb-docs-archive.librato.com/data_processing/composite_metrics/tutorial/ – Kotsu Nov 26 '18 at 14:48