Spring Boot Actuator 'http.server.requests' metric MAX time

Question

I have a Spring Boot application and I am using Spring Boot Actuator and Micrometer in order to track metrics about my application. I am specifically concerned about the 'http.server.requests' metric and the MAX statistic:

{
    "name": "http.server.requests",
    "measurements": [
        {
            "statistic": "COUNT",
            "value": 2
        },
        {
            "statistic": "TOTAL_TIME",
            "value": 0.079653001
        },
        {
            "statistic": "MAX",
            "value": 0.032696019
        }
    ],
    "availableTags": [
        {
            "tag": "exception",
            "values": [
                "None"
            ]
        },
        {
            "tag": "method",
            "values": [
                "GET"
            ]
        },
        {
            "tag": "status",
            "values": [
                "200", 
                "400"

            ]
        }
    ]
}

I suppose the MAX statistic is the maximum time of execution of a request (since I have made two requests, it's the the time of the longer processing of one of them).

Whenever I filter the metric by any tag, like localhost:9090/actuator/metrics?tag=status:200

{
        "name": "http.server.requests",
        "measurements": [
            {
                "statistic": "COUNT",
                "value": 1
            },
            {
                "statistic": "TOTAL_TIME",
                "value": 0.029653001
            },
            {
                "statistic": "MAX",
                "value": 0.0
            }
        ],
        "availableTags": [
            {
                "tag": "exception",
                "values": [
                    "None"
                ]
            },
            {
                "tag": "method",
                "values": [
                    "GET"
                ]
            }
        ]
    }

I am always getting 0.0 as a max time. What is the reason of this?

Romil Patel · Answer 1 · 2019-10-15T18:21:45.113

What does MAX represent (MAX Discussion)

MAX represents the maximum time taken to execute endpoint.

Analysis for /user/asset/getAllAssets

COUNT  TOTAL_TIME  MAX
5      115         17
6      122         17  (Execution Time = 122 - 115 = 17)
7      131         17  (Execution Time = 131 - 122 = 17)
8      187         56  (Execution Time = 187 - 131 = 56)  
9      204         56  From Now MAX will be 56 (Execution Time = 204 - 187 = 17)

Will MAX be 0 if we have less number of request (or 1 request) to the particular endpoint?

No number of request for particular endPoint does not affect the MAX (see an image from Spring Boot Admin)

When MAX will be 0

There is Timer which set the value 0. When the endpoint is not being called or executed for sometime Timer sets MAX to 0. Here approximate timer value is 2 to 2.30 minutes (120 to 150 seconds)

DistributionStatisticConfig has .expiry(Duration.ofMinutes(2)) which sets the some measutement to 0 if there is no request has been made for last 2 minutes (120 seconds)

Methods such as public TimeWindowMax(Clock clock,...), private void rotate() Clock interface has been written for the same. You may see the implementation here

How I have determined the timer value?

For that, I have taken 6 samples (executed the same endpoint for 6 times). For that, I have determined the time difference between the time of calling the endpoint - time for when MAX set back to zero

MAX property belongs to enum Statistic which is used by Measurement (In Measurement we get COUNT, TOTAL_TIME, MAX)

public static final Statistic MAX

The maximum amount recorded. When this represents a time, it is reported in the monitoring system's base unit of time.

Notes: This is the cases from metric for a particular endpoint (here /actuator/metrics/http.server.requests?tag=uri:/user/asset/getAllAssets).

For generalize metric of actuator/metrics/http.server.requests

MAX for some endPoint will be set backed to 0 due to a timer. In my view for MAX for /http.server.requests will be same as a particular endpoint.

UPDATE

The document has been updated for the MAX.

NOTE: Max for basic DistributionSummary implementations such as CumulativeDistributionSummary, StepDistributionSummary is a time window max (TimeWindowMax). It means that its value is the maximum value during a time window. If the time window ends, it'll be reset to 0 and a new time window starts again. Time window size will be the step size of the meter registry unless expiry in DistributionStatisticConfig is set to other value explicitly.

buræquete · Answer 2 · 2019-07-31T07:43:10.113

You can see the individual metrics by using ?tag=url:{endpoint_tag} as defined in the response of the root /actuator/metrics/http.server.requests call. The details of the measurements values are;

COUNT: Rate per second for calls.
TOTAL_TIME: The sum of the times recorded. Reported in the monitoring system's base unit of time

MAX: The maximum amount recorded. When this represents a time, it is reported in the monitoring system's base unit of time.

As given here, also here.

The discrepancies you are seeing is due to the presence of a timer. Meaning after some time currently defined MAX value for any tagged metric can be reset back to 0. Can you add some new calls to your endpoint then immediately do a call to /actuator/metrics/http.server.requests to see a non-zero MAX value for given tag?

This is due to the idea behind getting MAX metric for each smaller period. When you are seeing these metrics, you will be able to get an array of MAX values rather than a single value for a long period of time.

You can get to see this in action within Micrometer source code. There is a rotate() method focused on resetting the MAX value to create above described behaviour.

You can see this is called for every poll() call, which is triggered every some period for metric gathering.

Spring Boot Actuator 'http.server.requests' metric MAX time

2 Answers2

Linked