0

I am using GitLab and my test cases are written in C# Speflow. I have configured the pipeline to execute the complete automation suite before deployment but my pipeline does not fail and terminate on test case failure it keeps going and even when the complete suite is executed it's still not failing or stopping. The pipeline will only stop when the runner timeout limit is reached. Please see below image:

enter image description here

I want to stop the pipeline and mark it as failed as soon as any of the test cases fail. I used "allow_failure: false" but it didn't work. Can someone please help me with this? So in short I want to terminate the CI pipeline and fail it as soon as my first test case is failed and don't want my pipeline to continue with other remaining test cases.

My GitLab yml file is as below:

demo_job_1:
     stage: test

     tags:
       - tags
              
     script:
       - Root dir location path
       - 'dotnet restore <path to the ProjName.sln>'
       - 'dotnet msbuild <path to the ProjName.sln>'
       - 'dotnet test <path to the ProjName.sln>'

     allow_failure: false
Rashmi
  • 17
  • 7
  • Is the command returning any error when it fails? If some command return any code different than `0` it should makes the pipeline fail. – Adrian Dec 31 '21 at 12:30
  • I see the message "ChromeDriver was started successfully" on your screenshot. Does it stop automatically, though? Maybe it is the browser instance not stopping due to failed script. – xy2 Dec 31 '21 at 15:54
  • @xy2 Yes, it does stop the browser after every test case execution. – Rashmi Jan 04 '22 at 09:38

1 Answers1

0

The pipeline is giving you a hint, your test suite seems to be taking too long to execute all the tests.

Error: job failed: execution took longer than 2h0m0s seconds

First, try to debug what might be going on with your tests inside the pipelines

dotnet test --logger "console;verbosity=detailed"

Then, if possible, could you try running only a subset of your test suite? It might be that a single test might be hanging your entire pipeline.

Aristu
  • 761
  • 3
  • 17
  • 30
  • 1
    Thanks Aristu, but the pipeline execution was completed in 1 hour or less. As mentioned in my question, it waits for the runner's time out limit to reach, which is set to 2 hours. And after 2 hours even when there is no execution left, it gives the error msg saying "execution took longer than 2h0m0s seconds". – Rashmi Jan 04 '22 at 09:40
  • Hey Aristu, it took me a while to try your solution, but the command - dotnet test --logger "console;verbosity=detailed", didn't work in my case. I did not get any log after pipeline completion. Also, when u tried to run only certain classes as per your advice, gitlab didn't accept those commands and asked for the solution file. I was getting MSBuild error of not identifying the files, though my path was correct. – Rashmi Jan 10 '22 at 13:18
  • Could you please guide me for some other solution? I only want to terminate the pipeline on first test case failure itself and don't want it to continue executing other cases. Thankyou! – Rashmi Jan 10 '22 at 13:19
  • I don't think I'm able to help, unless you post more about your test cases, but I'm not well versed in C#. Your pipeline seems fine, the problem might be somewhere in the codebase (specially if it's full of `async/await` calls). – Aristu Jan 11 '22 at 10:57
  • Okay, thanks for your time – Rashmi Jan 12 '22 at 06:07