Postman always valid as true a response with the schema using tv4

Question

I've did a lot of tests but I coulnd't find the way to solve it. Simplifying, I have this Postman test script to validate that the JSON response es according with the JSON schema defined to this API:

const stockQuotesSchema = JSON.parse(pm.environment.get("schema"));

pm.test("Stock quote returns 200 OK", function () {
    pm.response.to.have.status(200);
})

pm.test("Stock quote is JSON", function () {
    pm.response.to.be.json;
})

pm.test("Stock quote response matches schema", function () {
    const validationResult = tv4.validateResult(pm.response.json(), stockQuotesSchema);
    pm.expect(validationResult.valid).to.be.true;
})

This is the API schema defined (simplified):

{
    "codexxx": "UNAUTHENTICATED",
    "messagexxx": "token expired"
}

This is the response I get after run the request:

{
    "code": "UNAUTHENTICATED",
    "message": "token expired"
}

As the "code" field and the "message" field doesn't exist in the schema I hope to get a FAIL, but I always retrieve a True.

This is the Postman result image

I need to validate every response with a long JSON schema (I mean bigger schema than the example above). Any idea? Thanks.

msbit · Accepted Answer · 2018-09-24T10:02:09.393

1

You can enforce the proposed (and then removed) banUnknownProperties mode through additional arguments to validateResult as described here. For example:

const tv4 = require('tv4');

const schema = {
    properties: {
        codexxx: {
          type: 'string'
        },
        messagexxx: {
          type: 'string'
        }
    }
};

const invalidResponse = {
    code: 'UNAUTHENTICATED',
    message: 'token expired'
};

const validResponse = {
    codexxx: 'UNAUTHENTICATED',
    messagexxx: 'token expired'
};

const invalidRelaxedResult = tv4.validateResult(invalidResponse, schema);
const validRelaxedResult = tv4.validateResult(validResponse, schema);
const invalidStrictResult = tv4.validateResult(invalidResponse, schema, false, true);
const validStrictResult = tv4.validateResult(validResponse, schema, false, true);

console.log('Relaxed (invalid):', invalidRelaxedResult.valid);
console.log('Relaxed (valid):', validRelaxedResult.valid);
console.log('Strict (invalid):', invalidStrictResult.valid,
            invalidStrictResult.error.message, invalidStrictResult.error.dataPath);
console.log('Strict (valid):', validStrictResult.valid);

which outputs:

Relaxed (invalid): true
Relaxed (valid): true
Strict (invalid): false Unknown property (not in schema) /code
Strict (valid): true

The third argument to validateResult specifies whether to do recursive checking. It's not immediately clear, but I believe that the default is false (as provided in the example above).

edited Sep 24 '18 at 10:02

answered Sep 21 '18 at 23:49

msbit

4,152
2
9
22

Thanks @msbit. Trying your code I get `Relaxed: ture` and `Strict: ` (empty). Supposing that empty is false, how can I know which part of the JSON is invalid? Because I have a very long JSON defining the API I should validate. – Sebastian Diaz Sep 24 '18 at 04:09
Hmm, you should be getting `false` instead of empty, perhaps take a look at the full value of `strictResult` (eg `console.log('Strict:', strictResult);`). For me it has `error` and `valid` fields (among others) and `error` has additional fields which detail the error (in this case `code` which is 1000 to indicate the unknown property and `dataPath` to indicate which property is unknown). – msbit Sep 24 '18 at 04:47
Ok, now taking a look at the full value of strictResult I see this: `Strict: error: { message:"Unknown property (not in schema)", name:"ValidationError" type:"Error"}, missing: [valid:false]`. So, it is answering "false" but it doesn't indicate what keyword is wrong. Another thing, if I put the same values of `const schema` into `const response`, I get exactly same error as well. It is very strange. I tried it in two last Postman version running on macOS High Sierra and Windows 10. – Sebastian Diaz Sep 24 '18 at 05:55
If you try with the same values in schema and response variables, or with differents, do you get differents results in Strict? – Sebastian Diaz Sep 24 '18 at 05:57
Ah, looks like the schema needs to be defined differently. I'm going to update the code in my answer with what I find. – msbit Sep 24 '18 at 06:40
@SebastianDiaz updated with the proper schema definition structure. – msbit Sep 24 '18 at 06:53
Thanks, that works as you indicate. I see that when tv4 find an invalid value it log that and don't continue with the remain checking. I'll continue preparing an script with this solution. Thanks again! – Sebastian Diaz Sep 25 '18 at 00:46

Postman always valid as true a response with the schema using tv4

1 Answers1