Reuse of feature files with retry until

Question

Having some stunning success with Karate. Working on end-to-end tests that use 'retry until' timeout on a GETs in order to wait for a specific parameter value within the response body. The parameter is expected to change state from A to B as data processing completes in the system under test. Interested to learn about patterns due to having standardised on a model of one .feature per API route. However this is only possible if we can parameterise the retry until terms. Otherwise this would mean writing multiple features to support different retry combinations.

---Example of reusing retry until from comments---

To maintain a single get_notification_ref.feature rather than one for each until combination, provide external until parameters in the call that can be used by the retry within the .feature.

Implementation that relies on specifying the until parameters inside the .feature file. End up with GET Notification feature file for each retry combination :

Scenario: Get notification & wait for status
   * call read('classpath:NotifyV1/get_notification_ref_wait_status.feature')
   .
   .

Scenario: Get notification & wait for status indicator colour
   * def expectedColour = 'GREEN'
   * call read('classpath:NotifyV1/get_notification_ref_wait_colour.feature')

get_notification_ref_wait_status.feature

Scenario: Get notification and wait on response status = 200
    Given path 'notification', notificationTypeReference
    And retry until responseStatus == 200
    When method get
    * def notificationResponse = $

get_notification_ref_wait_colour.feature

Scenario: Get notification and wait on response status = 200 and colour
    Given path 'notification', notificationTypeReference
    And retry until responseStatus == 200 && response.statusColour == expectedColour
    When method get
    * def notificationResponse = $

Implementation of the above that can handle parameterised retry until's would look like this - note there is now only one GET Notification feature file:

Scenario: Get notification & wait for status 200
   * call read('classpath:NotifyV1/get_notification_ref.feature')
   .
   .

Scenario: Get notification & wait for status 200 and indicator colour
   * def UntilTerm = function(response){ return karate.match(response, '{statusColour: "GREEN"}').pass }
   * call read('classpath:NotifyV1/get_notification_ref.feature')

get_notification_ref.feature

Scenario: Get notification
    * def untilTerm = karate.get('UntilTerm') ? UntilTerm : function(response){ return true }
    * def untilStatus = karate.get('UntilStatus') ? UntilStatus : 200
    Given path 'notification', notificationTypeReference
    And retry until responseStatus == untilStatus && untilTerm(response)
    When method get
    * def notificationResponse = $
    * karate.set('UntilTerm',null)
    * karate.set('UntilStatus',null)

score 1 · Accepted Answer · answered Nov 27 '19 at 10:00

1

I would say retry until may be sufficient. Since you can tweak the default time and interval, you can set this differently even at the point of need, i.e. a specific HTTP call: https://github.com/intuit/karate#retry-until

Unless you really truly have a way for the external process to call back - and in that case you can look at karate.signal() and friends. Otherwise I think you are better off sticking to retry until.

answered Nov 27 '19 at 10:00

Peter Thomas

54,465
21
84
248

1

Thank you. Will look at ways to keep the retry configs maintainable – mactwixs Nov 28 '19 at 17:22
Each of our API routes is in a separate .feature. We therefore do not want to have to have a variant of each API route .feature if a calling scenario needs 'retry until responseStatus ==200' only and another scenario needs both 'retry until responseStatus ==200 && response.colour == 'GREEN'. Is there latitude to compile the until terms prior to calling "retry until" e.g. bit like a query builder? – mactwixs Dec 02 '19 at 16:14
@mactwixs not sure I understand. you can use a pre-defined function if that helps: https://stackoverflow.com/a/55823180/143475 – Peter Thomas Dec 02 '19 at 17:54
have updated the original post with an example. My bad to try cramming it into a comment line. – mactwixs Dec 03 '19 at 06:26
Can you confirm how retry until treats null (if we were to parameterise the above example)? Thanks – mactwixs Dec 04 '19 at 07:34
@mactwixs sorry that is really hard for me to understand. in my above link please can you look for `And retry until isValid(response)` and the JS function defined - and really that is all you need IMO. write a JS function that takes a parameter or two by which you can drive the "variant" behavior. you can declare a JS function that is "global" to all features. Just do it :) Or please open a new question with a *specific* example. – Peter Thomas Dec 04 '19 at 07:37

Reuse of feature files with retry until

1 Answers1