2

I'm trying to run a simple test scenario on sauce-labs, but the issue is that sauce-labs doesn't identify the test-status as passed or failed on its own. I've gone through multiple solutions but couldn't make any of them work.

I would love to know if we can do something in the feature file rather than doing something externally if possible. Following are my feature and java files.

Feature File
1

Java File
2

Any help will be appreciated. I'm a JAVA newbie & just starting on the Karate Framework and do not have in-depth knowledge of how things are working.

Zephyr
  • 11,891
  • 53
  • 45
  • 80
Mohammad Hassaan Momin
  • 21
  • 2
  • no idea about sauce labs. I don't think they look at stack overflow, so maybe you should ask their support channel – Peter Thomas Aug 05 '20 at 12:54
  • @PeterThomas we actually do have a slack alert for saucelabs tagged questions on Stack Overflow! :) (though it is monitored by SA's not the support staff, so always better to go through support channels if there is an actual problem with our product). – titusfortner Aug 05 '20 at 17:03
  • @titusfortner great. I just scanned for all questions tagged `saucelabs` and didn't see too many answered especially the recent ones: https://stackoverflow.com/questions/tagged/saucelabs?tab=Newest – Peter Thomas Aug 05 '20 at 17:17
  • 1
    @PeterThomas good point, looks like our answer rate isn't as high as I thought it was. – titusfortner Aug 05 '20 at 17:30

1 Answers1

3

The Remote Driver is just sending along requests for actions and requests for information. What that information is "supposed" to be is decided by the assertions in your test. Sauce Labs has no way of knowing what that is without your input.

There are 2 ways to tell Sauce Labs the status of the test.

The recommended way is to use the API: https://wiki.saucelabs.com/display/DOCS/Job+Methods#JobMethods-UpdateJob You just need your sauce username, access key and the job ID (job ID is the same as the driver's session ID).

The less reliable, but easier to implement way is with a custom JavaScript command: https://wiki.saucelabs.com/display/DOCS/Annotating+Tests+with+Selenium%27s+JavaScript+Executor "sauce:job-result=passed"

Either of these can be put into an AfterClass or TestWatcher methods.

titusfortner
  • 4,099
  • 2
  • 15
  • 29
  • 2
    I would recommend the first option, you can experiment if within a test this works: `* karate.get('driver').script('sauce:job-result=failed')` - then the same may work in an `afterScenario()` hook - note that the recommended hook pattern is the `ExecutorHook`: https://stackoverflow.com/a/59080128/143475 – Peter Thomas Aug 05 '20 at 17:20
  • Thanks guys for all the help, I really appreciate it!! I was able to make it work with method# 1 @titusfortner mentioned. Here is what I did: 1. Created another feature file that will use the Sauce Labs REST API to update the job status using the PUT request `{ passed: true }` 2. Finally, in my main feature file I just update the the afterScenario function to call the newly created feature file respectively `if(info.errorMessage == null{karate.call('sauce-pass.feature')}` – Mohammad Hassaan Momin Aug 06 '20 at 14:27