My organization started to use Pact for creating/verifying contracts between REST services/micro services written in Java about half a year ago. We have a hard time deciding what the appropriate scope or grasp of a provider test should be and would love some input from the experience of other pact users out there.
Basically the discussion evolves around where to mock/stub in the provider tests. In a service you would have to mock external calls to other services at least but you have the option of mocking closer to the REST resource class as well.
We boiled it down to two options:
1. The first option is that a provider test should be a strict contract test and only exercise the provider service's REST resource class, mocking/stubbing out the service classes/orchestrators etc. used from there. This contract test would be augmented with component tests that would test the parts stubbed/mocked by the provider test.
2. The second option is to use the provider test as a component test that would exercise the entire service component for each request. Only transitive external calls to other components would be mocked/stubbed.
These are thoughts of pro's for each option
Pro's for option 1:
- Tests will be simpler to implement and will get a smaller footprint
=> higher degree of isolation. - We probably need other component tests anyway to cover use cases typically not covered in the consumer's expectancies (error flows etc). This way we wouldn't mix different kinds of component tests (Pact and other) in one bag, making the test suite easier to understand.
Pro's for option 2:
- Tests are exercising more of the "real" code => less risk for test bugs due to bad mocking/stubbing.
I would be really interested to hear how your provider tests typically look in this regard. Is there a best practice?
Clarifying what we mean by "Component": A component is a microservice or a module in a larger service application. We took the definition of 'component from Martin Fowlers http://martinfowler.com/articles/microservice-testing/.
A provider service/component typically has a REST endpoint in a Jersey resource class. This endpoint is the provider endpoint for a Pact provider test. An Example:
@Path("/customer")
public class CustomerResource {
@Autowired private CustomerOrchestrator customerOrchestrator;
@GET
@Path("/{customerId}")
@Produces(MediaType.APPLICATION_JSON)
public Response get(@PathParam("customerId") String id) {
CustomerId customerId = CustomerIdValidator.validate(id);
return Response.ok(toJson(customerOrchestrator.getCustomer(customerId))).build();
}
In the above example the @Autowired (we use spring) CustomerOrchestrator could either be mocked when running the provider test or you could inject the real "Impl" class. If you choose to inject the real "CustomerOrchestratorImpl.class" it would have additional @Autowired bean dependencies that in turn may have other... etc. Finally the dependencies will end up either in a DAO-object that will make a database call or a REST client that will perform HTTP calls to other downstream services/components.
If we were to adopt my "option 1" solution in the above example we would mock the customerOrchestrator field in the CustomerResource and if we were adopting "option 2" we would inject Impl-classes (the real classes) for each dependency in the CustomerResource dependency graph and create mocked database entries and mocked downstream services instead.
As a side note I should mention that we rarely actually use a real database in provider tests. In the cases where we adopted "option 2" we have mocked the DAO-class layer instead of mocking the actual database data to reduce the number of moving parts in the test.
We have created a "test framework" that automatically mocks any Autowired dependency that is not explicitly declared in the spring context so stubbing/mocking is a light weight process for us. This is an excerpt of a provider test that exercises the CustomerResource and initiates the stubbed CustomerOrchestrator bean:
@RunWith(PactRunner.class)
@Provider("customer-rest-api")
@PactCachedLoader(CustomerProviderContractTest.class)
public class CustomerProviderContractTest {
@ClassRule
public static PactJerseyWebbAppDescriptorRule webAppRule = buildWebAppDescriptorRule();
@Rule
public PactJerseyTestRule jersyTestRule = new PactJerseyTestRule(webAppRule.appDescriptor);
@TestTarget public final Target target = new HttpTarget(jersyTestRule.port);
private static PactJerseyWebbAppDescriptorRule buildWebAppDescriptorRule() {
return PactJerseyWebbAppDescriptorRule.Builder.getBuilder()
.withContextConfigLocation("classpath:applicationContext-test.xml")
.withRestResourceClazzes(CustomerResource.class)
.withPackages("api.rest.customer")
.build();
}
@State("that customer with id 1111111 exists")
public void state1() throws Exception {
CustomerOrchestrator customerOrchestratorStub = SpringApplicationContext.getBean(CustomerOrchestrator.class)
when(customerOrchestratorStub.getCustomer(eq("1111111"))).thenReturn(createMockedCustomer("1111111));
}
...
