API Testing containerized microservices with Karate

Question

I am trying to leverage Karate (https://github.com/intuit/karate) as a key component in an overall testing strategy for testing containerized, cloud-based microservices. Assuming that both the microservice under test and Karate have their own containers, the process is as follows:

1. Fetch each container for local deployment
2. Build (via gradle) the components in the Karate container (assume there are Java classes required by our mocks)
3. Deploy (via gradle) the mocks and get them running in standalone mode
4. Inject information about the mocks into the YAMLs of the microservice
5. Build and deploy the microservice locally
6. Run Karate tests (passing information about the mocks and/or environment) via CLI

My first question is whether or not this is a Good Idea(TM) or a Bad Idea(TM). On the surface it seems both reasonable and achievable but I am wondering if I am trying to use Karate in a way that it was never meant to be used. I toyed with the idea of keeping all the Karate stuff (including the mock sever) in the tests themselves, but then steps #3-5 would have to inject the mock information into the microservice, then run commands to get the microservice built and deployed all within test suite, which seemed to me like a Bad Idea(TM). Better instead to do this as part of a pipeline in a Jenkins job right?

My second question is how to best export mocks, files and Java dependencies for outside use (to support step numbers 2-3), for example here is the file structure:

.
+-- build.gradle
+-- src
|   +-- main
|       +-- java
|           +-- JWTSigner
|           +-- PEMHelper
|       +-- resources
|           +-- private-key.pem
|           +-- public-key.pem
+-- test
|   +-- main
|       +-- java
|           +-- api
|               +-- cats
|                   +-- cats.feature
|               +-- dogs
|                   +-- dogs.feature
|               +-- AllTestsRunner.java
|           +-- mocks
|               +-- mock-auth.feature
|           +-- templates
|               +-- public-key.json
|       +-- resources
|           +-- lolcats.pdf
|           +-- loldawg.jpg

So here, mock-auth.feature needs the stuff in src/main as well as in src/test/templates. I've been able to play around with gradle tasks and copy the stuff required into a subdirectory of a main directory with the standalone Karate JAR to so the mock can be started, but I was wondering if there's a better way...

Any feedback is appreciated, but if negative, please suggest an alternative for me to attempt. Thank you.

Answer 1

Answer to your second question is here: https://stackoverflow.com/a/58339662/143475

In my opinion, there is no Right Way™ to orchestrate your mocks and application-under-test. The approach you proposed will work, what I would do is use the --network option in Docker, for e.g. if the network is called mocks and you have to decide and perhaps expose a port number e.g. 8080, you set the URL-s in the second container as http://mocks:8080

You may get some ideas from this page on Karate's distributed-testing guide: https://github.com/intuit/karate/wiki/Distributed-Testing

Note that Karate mocks are designed to require only a JRE and the "fatjar". This can simplify things, for e.g. you may not even need to "Dockerize" things, just have java on the PATH and point to your *.feature files and you are done. Even if you use Java code, you can either add those to the CLASSPATH or you could build a "fatjar" yourself using maven-shade.

As you said, the option with the least moving parts may be to have the mocks stood-up in the same Java project. Advantages are:

1. you can dynamically choose a port and pass it to the main app config
2. starting and stopping and co-ordinating the 2 processes is easier (Java code)
3. CLASSPATH is automatically taken care of
4. if you get ambitious and attempt code-coverage in the future, can be easier

Further reading: https://stackoverflow.com/a/61414022/143475