0

I have followed How to Run Karate API tests on Azure pipelines to set up karate on azure pipeline.

We need to first start up the application in order to run karate tests.

  1. Imagine I have a dev api publicly available and I have some new changes to merge in.

    It still doesn’t make sense to test on this dev environment because the new karate test cases on the new features are not yet available on dev and of course the karate tests will fail.

  2. If we look at example projects they all excluded the karate test files. What is the reason to exclude them? Shouldn't we include them instead so the karate test run during pipeline?

    <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.22.1</version>
    <configuration>
        <excludes>
            <exclude>karate/**/*.java</exclude>
        </excludes>
        <includes>
            <include>com/**/*.java</include>
        </includes>
    </configuration>
</plugin>
James Z
  • 12,209
  • 10
  • 24
  • 44
BabyishTank
  • 1,329
  • 3
  • 18
  • 39
  • I don't understand. is this a `karate` question ? or is the question "how do I start a java application" on azure ? and isn't the answer "just call a `main` method from the command line" ? – Peter Thomas Feb 18 '22 at 07:49
  • @PeterThomas, this is more of a azure script question. The Maven@3 task is azure's way of starting an application. I want to know what enviornment others use to run their karate test. They should also run into question 1. – BabyishTank Feb 18 '22 at 15:55
  • There's too many different questions being asked here. You mention building your application, which env to start testing in, and ask why `.java` files are excluded. How will anyone know where to start? – anutter Feb 22 '22 at 17:13

2 Answers2

2

you can use maven command in azure to run karate test as I understood your requirement correctly.

But if you need to different steps for building application and running test, you need to have two step in the pipeline in order, like creating bash script or something similar. Maybe you can use Makefile and make command to build and run your test, like;

build-and-run:
     mvn (run app command) && mvn clean test -Dtest=myRunner

Currently I am using azure devops for running some karate tests but my tests doesn't required to have make application since they are run against staging env. I used this step for running karate test:

    - task: Maven@3
  inputs:
    mavenPomFile: 'pom.xml'
    goals: 'clean test -Dusername=$(USERNAME) -Dpassword=$(PASSWORD)'
    publishJUnitResults: true
    testResultsFiles: '**/surefire-reports/TEST-*.xml'
    javaHomeOption: 'JDKVersion'
    jdkVersionOption: '1.8'
    mavenVersionOption: 'Default'
    mavenOptions: '-Xmx3072m'
    mavenAuthenticateFeed: false
    effectivePomSkip: false
    sonarQubeRunAnalysis: false

This is autogenerated by azure devops, I just put my run command

Selcuk
  • 31
  • 2
  • 1
    thank you, may I know if there is a maven-surefire-plugin in your pom.xml and did you "excluded" the karate java files? – BabyishTank Feb 18 '22 at 22:14
  • Yes I have maven surefire plugin and excluded tests. But excluding tests won't causes any problem while you are running your tests IMHO. – Selcuk Feb 21 '22 at 10:31
  • unless you also have failsafe plugin, otherwise your karate tests are ignored in the pipeline – BabyishTank Feb 21 '22 at 17:34
  • 1
    I don't have failsafe plugin but I am not running spring application. I just run my karate tests. Not sure for spring since I don't have spring app running – Selcuk Feb 22 '22 at 05:38
1

Well.. none of you were able to mention the maven failsafe plugin which were designed to run integration test.

First start up the enviornment with spring-boot-maven-plugin

This will start up your spring application in the azure hosted agent to run the karate test.

Karete documentation did a good job explaining how to change the enviornment in karate config.js. But didn't mention about what enviornment we should use in pipeline

It would still be localhost- to the virtual machine starting your spring application in pipeline

BabyishTank
  • 1,329
  • 3
  • 18
  • 39
  • 1
    maybe you confused karate with maven :) anyway, that's what stack overflow is for. glad you found a solution and this thread will help others in the future. you rock ;) – Peter Thomas Feb 21 '22 at 06:20