1

Is there a way to get Karate to automatically print the name of each scenario as it is executed into the logs? We have a reasonably large suite that generates ~25MB of log data in our Jenkins pipeline console output and sometimes it’s a little tricky trying to match a line where com.intuit.karate logs an ERROR to the failure summary at the end of the run. It is most likely possible to obtain the scenario name and print() it but that would mean adding code to many hundred scenarios which I’d like to avoid.

Nemgathos
  • 605
  • 1
  • 5
  • 13
Dunc
  • 83
  • 1
  • 3

1 Answers1

0

As part of the fix for this issue Karate will log the Scenario name (if not empty) along with any failure trace.

A beta version with this fix is available 0.6.1.2 it would be great if you can try it and confirm.

If you feel more has to be done, do open a ticket and we'll do our best to get this into the upcoming 0.6.2 release.

Peter Thomas
  • 54,465
  • 21
  • 84
  • 248
  • 1
    Fantastic, cheers Peter I'll give that a try and let you know how I get on! – Dunc Nov 29 '17 at 15:31
  • Does the scenario name only get logged on failure and would you expect the scenario name to be logged inline with the requests / responses or in the Failed scenarios section at the end of a .feature file run? – Dunc Nov 29 '17 at 15:59
  • e.g. T E S T S ------------------------------------------------------- Running examples.ExamplesTest *** Is there any way to get the Scenario name to be logged at this point? *** 15:52:26.232 [main] INFO com.intuit.karate - karate.env system property was: production 15:52:26.664 [main] DEBUG com.intuit.karate - 1 > GET https://jsonplaceholder.typicode.com/doesntexist etc. – Dunc Nov 29 '17 at 16:00
  • @Dunc I can see how that can be useful, I've also felt the pain of wading though large logs. BUT - have you ever tried the HTML reports as seen here: https://github.com/intuit/karate#test-reports - most teams use this, and troubleshooting is much easier. the main problem you must be facing is distinguishing between the parallel threads which the HTML solves. else please do open a feature request and I will look at options. – Peter Thomas Nov 29 '17 at 16:24
  • 1
    We have tried to use the HTML reports which for shorter runs are excellent - however the system we're testing has well over a thousand discreet scenario runs and they get very big very quickly and Chrome and even Firefox Quantum start to struggle : ) One side effect of the ease of use of Karate is that we've generated _way_ more tests (and as a result way better coverage) than earlier solutions - it's almost *too* good! – Dunc Nov 29 '17 at 16:33
  • 1
    I should correct myself - we've tried the cucumber reports, not those linked. I'll add that to our build and give it a try. Anything that aids our ability to quickly diagnose faults is great. – Dunc Nov 29 '17 at 16:40
  • @Dunc great. this short video shows what to expect: https://twitter.com/KarateDSL/status/899671441221623809 – Peter Thomas Nov 29 '17 at 16:52