25

We're required to use the API of an external partner. The API is in a good shape and we got access to a sandbox environment we can use for automatic testing.

We already test every single call of the external API using unit tests but are unsure regarding best practices for integration tests when it comes to complex operations on the side of the external partner.

Example: Every user of our service also got a user object at our external partner. When performing external API call X on this user object, we expect object Y to appear inside collection Z of this user (which we have to query using a different call).

What are best practices for testing cases like this?

  • Mock the external API as much as possible and rely on the unit tests to do their job? Advantages: Tests run fast and independent from an internet connection. Disadvantages: Mistakes is in our mocks could lead to false positives.

  • Integrate the external API sandbox and run every integration test against it. Advantages: Close to real life API interactions. Disadvantages: Tests can only be run with an open internet connection and take more time.

  • Use a hybrid of mocked and sandbox data, set a boolean to switch between the internal (=mocked) and external (=sandbox) environment when required. Advantages: Reliable tests. Disadvantages: Could be a pain to set up.

  • Other best practices?

Thanks!

Related: How are integration tests written for interacting with external API? However, the answer "You don't. You have to actually trust that the actual API actually works." is not sufficient in our opinion.

[EDIT] We fear that integration testing only against our assumptions how the external API should work (even if they are based on unit tests) – and not against the actual API – will leave us with false positives. What we'd need is a test that verifies that our assumptions (mocks) are actually correct – not only in the context of unit tests but also in the context of complex operations with several steps.

Validation might be a good example: What if we mess up the integration code and send malformed data or data that does not make any sense in the context we send it in because we missed a step? Our mock API, which does not validate (or only in very limited range) would still return valid data instead of passing the error we would receive from the real API.

Community
  • 1
  • 1
zerodot
  • 805
  • 1
  • 7
  • 15
  • You do not explain *how* the answer is insufficient. So we do not know why that abswer dies not apply in your case, and so do not know how to a swer your question. – Raedwald Jan 22 '15 at 00:23
  • possible duplicate of [How are integration tests written for interacting with external API?](http://stackoverflow.com/questions/7564038/how-are-integration-tests-written-for-interacting-with-external-api) – Raedwald Jan 22 '15 at 00:24
  • Thanks, you're right, I should have elaborated more. I've just expanded the last section with more context. – zerodot Jan 22 '15 at 06:48

1 Answers1

20

I believe there should be 2 level of verifications we need to do when we interface with an external API:

  • API verification: verify that the API works according to its specs and/or our understanding
  • App functionality verification: verify that our business logic works according to the expectation to the API that passes API verification

In our case, we use a mock API together with real and mock API verification.

  • Mock API allows us to isolate any runtime errors/exceptions to app functionality only, so we don't blame any external party for issues
  • The same API verification is executed against both real and mock APIs, to make sure that the real one works the way we expect, as well as the mock one should mimic the real one correctly

If along the way, external API changes, API verification may turn red, triggering changes in mock API. Changes in mock API may make app verification turn red, triggering changes in app implementation. This way you never miss any gap between external API and app implementation (ideally).

Another extra benefit of having a mock API + API verification is that your developers can use it as a documentation/specification of how the API is supposed to work.

hidro
  • 12,333
  • 6
  • 53
  • 53
  • 2
    Thanks, sounds like a good approach. Is this answer only based on your experience or are there external resources you could reference? – zerodot Jan 22 '15 at 06:55
  • It's actually what we are doing right now to test our Android apps that use external APIs. For mock server we use [sinatra](http://www.sinatrarb.com/) for its simplicity. For API verification, we use combination of [cucumber](http://cukes.info/) and [rest-client](https://github.com/rest-client/rest-client) to create BDD API tests. We used to use a sandbox server for testing, but it's shared by many other teams, which usually results in staled test data. – hidro Jan 22 '15 at 07:01
  • Thanks. Consulted a few other devs on this matter and got similar feedback, will mark this answer as the solution. – zerodot Jan 23 '15 at 09:28