1

Is it possible to continue the execution of test step even if one of the assert/match fails?

Ex:

Scenario: Testing
* def detail = {"a":{"data":[{"message":["push","dash"]},{"message":["data","Test"]}]}}
* match detail contains {"a":{"data":[{"message":["push","dash"]}]}}
* print detail

Here match will fail but execution stop at that point. is there a way to do a soft assertion so that next step gets executed?

Rahul
  • 49
  • 1
  • 6

1 Answers1

5

EDIT in 2021 - a PR introducing a continueOnStepFailure flag was contributed by Joel Ramos here and is available in Karate 1.0 onwards. You can find more details here: https://stackoverflow.com/a/66733353/143475

If you use a Scenario Outline each "row" is executed even if one fails.

Scenario Outline: Testing
* def detail = { a: 1, b: 2, c: 3 }
* match detail contains <expected>

  Examples:
    | expected |
    | { a: 1 } |
    | { b: 2 } |
    | { c: 3 } |

Note that the concept of "soft assertions" is controversial and some consider it a bad practice:

a) https://softwareengineering.stackexchange.com/q/7823

b) https://martinfowler.com/articles/nonDeterminism.html

For those looking for a way to show all mis-matches between 2 JSON objects, see this: https://stackoverflow.com/a/61349887/143475

And finally, since some people want to do "conditional" match logic in JS, see this answer also: https://stackoverflow.com/a/50350442/143475

Peter Thomas
  • 54,465
  • 21
  • 84
  • 248
  • Hi Peter do you have any plan to implement soft assertion with karate ? It will be extremely helpful specially in Ui Automation.? – Vikas Thange Mar 31 '20 at 18:56
  • 1
    @VikasThange nope, I'm not convinced it is needed. very few people have ever asked for this including you. there are ways to do it yourself if you write a JS or Java function. and of course, you are welcome to submit a PR :) – Peter Thomas Mar 31 '20 at 19:19
  • 1
    update in 2021 - Karate 1.0 will support a `configure continueOnStepFailure` flag, and I have edited my answer to include this option – Peter Thomas Mar 06 '21 at 08:32
  • Soft assertions allow the ability to provide higher diagnostic output when multiple assertions fail. For example, if I'm needing to confirm that certain values exist in an output, soft assertions mean I can report about all that aren't available -- not just the one that happens to be first in my list of match statements. With a scenario outline, the test has to execute individually for every single criteria. That's redundant, slow, unnecessary, and wasteful -- especially if I have to make a separate API call each time. – Keith Tyler Feb 28 '22 at 21:06
  • @KeithTyler please contribute code to get soft-assertions working :) – Peter Thomas Mar 01 '22 at 05:00