How to assert a JSON response which have results in random order every time in JMeter?

Question

I am using JSON Assertion to assert if a JSON path exists. Suppose I have a JSON response of an array of 'rooms' that 'contains' an array of cabinets, just like the following example

"rooms":
[
    {
        "cabinets":
        [
            {
                "id":"HFXXXX",
                "locationid":null,
                "name":"HFXXXX",
                "type":"Hosp"
            },
            {
                "id":"HFYYYY",
                "locationid":null,
                "name":"HFYYYY",
                "type":"Hosp"
            },
            {
                "id":"HFZZZZ",
                "locationid":null,
                "name":"HFZZZZ",
                "type":"Hosp"
            }
        ],
        "hasMap":false,
        "id":"2",
        "map":
        {
            "h":null,
            "w":null,
            "x":null,
            "y":null
        },
        "name":"Fantastic Room#3"
    }
],
[
    {   "cabinets":
        [
            {
                "id":"HFBBBB",
                "locationid":null,
                "name":"HFBBBB",
                "type":"Hosp"
            }
        ],
        "hasMap":false,
        "id":"3",
        "map":
        {
            "h":null,
            "w":null,
            "x":null,
            "y":null
        },
        "name":"BallRoom #4"
    }
]

I want to Make sure that the 'id' of all the cabinets are correct, therefore I define the JSON path as rooms[*].cabinets[*].id and expect the value to be ["HFXXXX","HFYYYY","HFZZZZ","HFBBBB"]

This works perfectly except that sometimes the values are returned in a different order["HFBBBB", "HFXXX","HFYYYY","HFZZZZ"] instead of ["HFXXXX","HFYYYY","HFZZZZ","HFBBBB"], hence the assertion will fail. The problem is with the order of the returned array and not the values themselves.

Is there a way to sort the order of a response before Asserting and keep using the JSON assertion? or the only way of doing this is extracting the value i want to assert against and use it in JSR223 Assertion (groovy or javascript)? if that is the case can you show me an example of how I could do it in JSR223 plugin.

The order of json arrays is maintained. If your tests fail because the order changed, then there is likely an error. The order will be defined by the backend you are calling. If you were to sort your json array you would break the response and your test wouldn't really test anything — pandaadb, Nov 13 '18 at 17:13
@pandaadb - True, the order is defined by the backend I am calling. In the example above I tried to keep it simple to keep the problem simple. But in some cases, if you have multiple rooms for example that have multiple cabinets, they could return in a different order for many reasons. — Ali Mahdi, Nov 14 '18 at 07:47
If two calls return the same set of results in different ordered then there is a problem with your backend surely. The Json in the arrays should always maintain the order that it was given by your backend. What i mean by that is that your failures to me look like legit failures on the backend. Why would the same call ever return 2 different orders of rooms? — pandaadb, Nov 14 '18 at 10:18
I see your point, however, I get different results if you perform an edit that manages to change some values, then the returned JSON results might be in a different order. Are you saying that even after an update or changing some values, the JSON should still maintain its order? — Ali Mahdi, Nov 14 '18 at 12:08
If you do an update that changes the order than this would be reflected in the call, yes. But then asserting by resorting the values won't be a good test I think? Since you are then not asserting that your update has changed the order at all. SO i think if you expect an order change, you need to assert the order change as well? There is some more info about order of json arrays here: https://stackoverflow.com/questions/7214293/is-the-order-of-elements-in-a-json-list-preserved/7214312 — pandaadb, Nov 15 '18 at 13:20

score 1 · Accepted Answer · answered Nov 14 '18 at 08:08

I would recommend using a dedicated library, for instance JSONAssert, this way you will not have to reinvent the wheel and can compare 2 JSON objects in a single line of code

Download jsonassert-x.x.x.jar and put it somewhere to JMeter Classpath
Download suitable version of JSON in Java library and put it to JMeter Classpath as well. If you're uncertain regarding what is "JMeter Classpath" just drop the .jars to "lib" folder of your JMeter installation
Restart JMeter so it would be able to load the new libraries
Add JSR223 Assertion as a child of the request which returns the above JSON
Put the following code into "Script" area:
```
def expected = vars.get('expected')
def actual = prev.getResponseDataAsString()

org.skyscreamer.jsonassert.JSONAssert.assertEquals(expected, actual, false)
```
It will compare the response of the parent sampler with the contents of ${expected} JMeter Variable, the order of elements, presence of new lines, formatting do not matter, it compares only keys and values

In case of mismatch you will have the error message stating that as the Assertion Result and the full debugging output will be available in STDOUT (console where you started JMeter from)

Thank you so much, I will try it. – Ali Mahdi Nov 14 '18 at 08:27 — Ali Mahdi, Nov 14 '18 at 08:27

How to assert a JSON response which have results in random order every time in JMeter?

1 Answers1