1

I'm looking for best practices or the recommended approach to test async code execution with Karate.

Our use cases are all pretty similar but a basic one is:

  1. Client makes HTTP request to API
  2. API accepts request and creates a messages which is added to a queue
  3. API replies with ACCEPTED / 202
  4. Worker picks up message from queue processes it and updates database
  5. Eventually after the work is finished another endpoint delivers updated data

How can I check with Karate that after processing has finished other endpoints return the correct result?

Concrete real life example:

  1. Client requests a processing intensive data export to API e.g. via HTTP POST /api/export
  2. API create message with information for creating the export and puts it on AWS SQS queue
  3. API replies with 202
  4. Worker receives message and creates export, uploads result as ZIP to S3 and finally creates and entry in the database symbolizing this export
  5. Client can now query list exports endpoint e.g. via HTTP GET /api/exports
  6. API returns 200 with the list of exports incl. the newly created entry

Generally I have two ideas on how to approach this:

  1. Use karate retry until on the endpoint that returns the list of exports
  2. In the API response (step #3) return the message ID and use the HTTP API of SQS to poll that endpoint until the message has been processed and then query the list endpoint to check the result

Is either of those approach recommended or should I choose an entirely different solution?

steve1337
  • 137
  • 1
  • 8

1 Answers1

1

The moment queuing comes into the picture, I would not recommend retry until. It would work if you are in a hurry, but if you are ok to write a little bit of Java code, please read on. Note that this Java "glue code" needs to be written only once, and then the team responsible for writing the functional flows will be up and running.

I personally would prefer option (2) just because when a test fails, you will have a lot more diagnostic information and traces to look at.

Pretty sure you won't have a problem using AWS Java libs to do things such as polling SQS.

I think this example will answer all your questions: https://twitter.com/getkarate/status/1417023536082812935

Peter Thomas
  • 54,465
  • 21
  • 84
  • 248
  • Just one question. I'm more than comfortable writing Java code. However our backend is written in node.js and to run karate tests we are using the official npm package: https://www.npmjs.com/package/@karatelabs/karate. Can you please point me to some docs how I can use Java interop in that case? – steve1337 Nov 09 '22 at 19:13
  • 1
    @steve1337 I'll be completely transparent, the npm option is still the java-flavored karate, with an embedded JVM. the intent for that is to just make it easier to introduce karate into hard-core node shops. you would get the same experience if you install a JRE and use the "fat jar" for eg: https://github.com/karatelabs/karate/tree/master/karate-netty#standalone-jar that said, please look at the option of calling CLI or OS processes, even batch scripts. my hunch is - that may be the route you want to go to bridge karate tests and your node code: https://stackoverflow.com/a/73321719/143475 – Peter Thomas Nov 10 '22 at 04:11