0

I am running C#/Selenium tests in Jenkins via a batch file.

Example call "C:\FrontEnd\MobileRegression.bat"

And inside the bat file

c:\SpecRun\SpecRun.exe run Default.srprofile /baseFolder:C:\testBase /filter:"@MobileGalaxy" /log:specrun.log /outputfolder:output /report:MobileGalaxy.html

This works fine and in this examplw there are 10 scenarios to be run. Sometimes 1 or 2 of the scenarios fail but they are set to rertry twice more if this happens and always they pass on the 2nd or 3rd try. So at the end of the run the result is a Pass and because some of the scenarios failed on their first run Jenkins logs this as a fail. Is there anyway of changing this (for example if 90% of tests pass then mark as PASS)?

Thanks in advance for any help

Kev

Kev
  • 121
  • 3
  • 20
  • Are you using a pipeline script, declarative pipeline or just a simple shell execution? – JRichardsz Nov 02 '18 at 14:22
  • It is a simple shell execution using the 'Execute Windows batch command' Build Step. – Kev Nov 02 '18 at 15:03
  • Are your 10 scenarios inside one batch or several batch files? – JRichardsz Nov 02 '18 at 15:52
  • The scenarios are in one batch which had the command [c:\SpecRun\SpecRun.exe run Default.srprofile /baseFolder:C:\testBase /filter:"@MobileGalaxy" /log:specrun.log /outputfolder:output /report:MobileGalaxy.html]. Each scenario has the tag @MobileGalaxy – Kev Nov 05 '18 at 07:15

1 Answers1

1

Jenkins ends with error because your batch file return an exit code different to zero, which means:

When a program finishes executing it returns an exit code to the system. The exit code (also called "exit status") is an integer from 0 to 255. The batch system reports this exit code. Always zero to success an non-zero to error.

According to the comments, I suggest to you, the following approaches:

#1 try catch inside your batch file

You need to ensuring that your batch ends with zero as exit code or success. You could use errorlevel and exit /b 0.

Here some theory and examples:

#2 try catch using jenkins pipeline script

In this approach you need to split your one batch file (with 10 scenarios) to 10 batch files (one by scenario)

After that, you could use jenkins pipeline script to invoke each batch file and using a try catch , control the error and also update a counter_error variable.

At the end of your jenkins script, you could evaluate :

if count_error > 9
  // notify a success execution to jenkins 
  // using currentBuild.result = 'SUCCESS'

Here some theory and examples:

JRichardsz
  • 14,356
  • 6
  • 59
  • 94
  • @kev if this helped to solve your day, please click the checkmark on the left to mark this as solved or just mark as useful :D – JRichardsz Nov 06 '18 at 14:07