1

Consider for example we have scenarios that test basic CRUD operations for rows in a specific relational database table (using endpoints from a web server). We then have another scenario that returns all of the rows in that table, using an endpoint called 'getAll' or something to that effect. Certainly in karate we can verify that the 'shape' of the response is how we expect using something like

And match response[*] == #something

But is there a way to actually verify that we get back the expected number of rows? In short, is there a way to strictly control our test data so that we know precisely what is in the database table at the beginning of the test? Certainly we could do this using Java interop if we could run a scenario atomically, but as far as I know there is currently no way to do this. And, if we are not running atomically, due to the parallel nature of karate tests (by default), I don't see a way to ensure that we know the state of the database throughout an entire test, without running all of the karate scenarios sequentially.

Techmaster21
  • 23
  • 5

2 Answers2

1

In case you haven't seen the dogs.feature in the demos: enter image description here

But looks like you have and you are aware of Java interop. The question you ask is interesting, it is the first time I heard of the parallel execution capability being a "problem".

Normally teams I have seen write tests that are scoped to a single entity (e.g. customer, order) that was created in the same Scenario therefore not dependent on any other test in your suite.

Also, nothing stops you from setting number of threads to 1.

And do be aware that within a Feature you can request all Scenario-s to run in sequence: https://github.com/intuit/karate#parallelfalse

Anyway if you really want to go down this path, here's what I suggest. You can write a Java helper to lock your database, run a few steps and unlock, how you do this is up to you.

Also note that karate.callSingle() will actually lock across all threads. This is normally used to do a "set up" routine, such as get an auth token for all your tests. In your case you could take a "snapshot" of your DB so that your tests can derive expected results. But you can experiment with it if it helps.

Otherwise please assume that Karate doesn't support what you are asking for. Note that splitting a logical "flow" into multiple "Scenario-s" is something I thoroughly discourage: https://stackoverflow.com/a/46080568/143475

Peter Thomas
  • 54,465
  • 21
  • 84
  • 248
1

Your data in DB should have some kind of segmentation e.g. data for user-1 would not not overlap data for user-2.

In this case assign a segment of data to a scenario e.g. user-1 So you can have these steps in the SAME scenario

  • Delete all the data for user-1
  • Add some data for user-1
  • verify data added for user-1
  • update added data for user-1
  • verify data updated for user-1
  • delete ...

Another scenario can run tests with data for user-2 - so these scenarios can now run in parallel.

Implementation wise, what I found very useful is a feature in karate to "call scenario" from another feature file ( https://intuit.github.io/karate/#calling-other-feature-files ). So I have, what I call "callable scenarios" in one feature file, to actually do the stuff in the line items in the test above. The "test scenario" calls the feature in "callable scenario" to delete data, add data, update data ...., and even to verify the data. The verify would do a "select" from the DB for the userid, and match against the expected data.

e.g. file crud-callable.feature has the following :

@delete-all
# Caller in "test scenario" can call this "callable scenario" with a syntax like : 
#     call crud-callable.feature@delete-all {userId: userId}
Scenario: Delete all for a user
  * def userId = karate.get ('userId', 'testUserId1')
  # delete data for userId
  ...

@add-data
Scenario: Add data for a user
   ...

@update-data
Scenario: Update data for a user
   ...

...

During development, the callable scenarios can be "unit tested" by running them manually, when they will use the hardcoded userId e.g. 'testUserId1'. When called from an automated test - they will use the userId passed in by the caller. Then these "callable scenarios" can be used from multiple "test scenarios", with each "test scenario" passing a different userId to the "callable scenario"

Now if we have different test scenarios use different userIds (or different segments of data, depending how your data is segmented in the database) - they can run be in parallel. Unless of course, you have a trivial system that does not allow multiple users to use the system simultaneously without causing issues.

Ari Singh
  • 1,228
  • 7
  • 12