End Gatling simulation when scenario fails BUT generate a report

Question

I have code which currently will not run my scenario if it fails;

//Defined outside of the scenario scope
var simulationHealthy = true

//defined within the scenario
.exec((session: io.gatling.core.session.Session) => {
  if (session.status == KO) {
      simulationHealthy = false
  }
    session
  })

However my simulation keeps running until the duration set for the simulation is over, though the scenario will not keep executing.

What I would like to do is to have a scenario fail under conditions I define (similar to assertions) and for the entire simulation to fail at that point as well, and also generate a report.

Thanks

Edit: I am running these tests within the IntelliJ IDE. Ending the simulation programmatically is required.

By default, Gatling generates a report even when it is killed via `Ctrl-C` in console. So, you can try to use `Runtime.getRuntime().exec(kill ....)` call to imitate a `Ctrl-C` action. You might want to check this answer: http://stackoverflow.com/questions/4750470/how-to-get-pid-of-process-ive-just-started-within-java-program — xeroqu, Dec 06 '16 at 09:25
@xeroqu Hi Xeroqu, I did discover this recently (after posting the question) however the main thing I want to be able to do here is end the simulation programmatically. Also I am running my simulations from within the IntelliJ IDE and am unsure how to generate the reports when I end the simulation within the IDE. — Questioning, Dec 06 '16 at 10:23

Gerald Mücke · Answer 1 · 2016-12-07T15:18:00.417

You might run the test itself without report and produce the report with a second call for just the report generation from the simulation.log

Run Simulation w/o report (-nr flag), i.e.

gatling.sh -nr -s YourSimulationClass

Generate Report (-ro flag):

gatling.sh -ro yoursimulation

(your simultation is the path underneath the results folder, which can be specified with -rf, which contains the simulation.log file)

In IntelliJ you can define another LaunchConfiguration to be executed before. So you define an action for executing Gatling Test (with -nr flag) and another configuration for report generation (with -ro flag), that executes the Gatling Test run action before.

Alternatively you could use the gatling-maven-plugin and define two executions (run, report) with the same flags.

Edit

According to this group thread you could execute your steps conditionally or mute them. The condition could be the presence of an error, but anything else as well. If the condition depends on global state i.e. a global variable, it would mute all users (unlike exitHereIfFailed)

For example:

val continue = new AtomicBoolean(true)
val scn = scenario("MyTest")
  .exec( 
    doIf(session => continue.get) {
      exec(http("request_0").get("/home").check(status.is(200)))
     .exec((session: io.gatling.core.session.Session) => {
       if (session.status == KO) {
         continue.set(false)
       }
       session
     })
  })

As said, this only stops sending requests to the SUT. Seems there is no other option at the moment (apart from System.exit(0))

Thanks, however I cannot mark this as the answer as it does not contain a way to end the simulation on error. — Questioning, Dec 06 '16 at 10:55

score 5 · Answer 2 · answered Jun 19 '18 at 11:44

You can use exitHereIfFailed in ScenarioBuilder returned by exec().

.exec(http("login")
    .post("/serviceapp/api/auth/login")
    ...
    .check(status.is(200))))
.exitHereIfFailed
.pause(1)
.exec(http("getProfileDetails")
      .get("/serviceapp/api/user/get_profile")
      .headers(authHeader("${token}"))
      .check(status.is(200)))

score 2 · Accepted Answer · answered Dec 09 '16 at 07:06

Thanks to @GeraldMücke 's suggestion of using system.exit I've come up with a work around. Still no where close to ideal but it does the job.

The problems are

Still have to manually generate the report from the log that is created when gatling is run
The user has to constantly manage how long the scenario lasts for both items as I don't know a way to have a scenario last the length of a simulation
This is obviously a "proof of concept" it has nothing in the code to define failure over thresholds etc like the asserts and checks available in Gatling itself

Here's the code. I've nested simulations within the setUp function because it fits the criteria of the work I am doing currently, allowing me to run multiple simulations within the main simulation.

FailOverSimulation and ScenarioFailOver are the classes that need to be added to the list; obviously this only adds value when you are running something that loops within the setUp.

import java.util.concurrent.atomic.AtomicBoolean

import io.gatling.commons.stats.KO
import io.gatling.core.Predef._
import io.gatling.core.scenario.Simulation
import io.gatling.http.Predef._
import scala.concurrent.duration._

object ScenarioTest {
  val get = scenario("Test Scenario")
    .exec(http("Test Scenario")
      .get("https://.co.uk/")
    )
    .exec((session: io.gatling.core.session.Session) => {
      if(session.status == KO) {
        ScenarioFailOver.exitFlag.set(true)
      }
      session
    })
}

object TestSimulation {
  val fullScenario = List(
    ScenarioTest.get.inject(constantUsersPerSec(1).during(10.seconds))
  )
}

object ScenarioFailOver {
  var exitFlag = new AtomicBoolean(false)

  val get = scenario("Fail Over")
    .doIf(session => exitFlag.get()) {
      exec(s => {
        java.lang.System.exit(0)
        s
      })
    }
}

object FailOverSimulation {
  val fullScenario = List(
    ScenarioFailOver.get.inject(constantUsersPerSec(1).during(10.seconds))
  )
}

class SimulateTestEnding extends Simulation {
  setUp(
    FailOverSimulation.fullScenario
      ::: TestSimulation.fullScenario
  ).protocols(
  )
}

End Gatling simulation when scenario fails BUT generate a report

3 Answers3

Edit

Linked