I use dotnet core with xUnit for my unit tests and also for my integration tests. I have a base abstract class for all my tests following the Given-Then-When philosophy in this way:
namespace ToolBelt.TestSupport
{
public abstract class Given_WhenAsync_Then_Test
: IDisposable
{
protected Given_WhenAsync_Then_Test()
{
Task.Run(async () => { await SetupAsync();}).GetAwaiter().GetResult();
}
private async Task SetupAsync()
{
Given();
await WhenAsync();
}
protected abstract void Given();
protected abstract Task WhenAsync();
public void Dispose()
{
Cleanup();
}
protected virtual void Cleanup()
{
}
}
}
As a summary, for each fact (then), the constructor (given) and the action (when) are executed again. This is very interesting to achieve idempotent tests because every fact should be runnable in isolation (the given should be idempotent). This is great for unit tests.
But for integration tests sometimes I find issues in scenarios like this:
I have a mongoDb repository implementation I want to test. I have tests to verify that I can write on it, and others to verify that I can read from it. But since all these tests run in parallel I have to be very mindful of how to setup the Given
and how and when to clean the context.
Test class A:
- Given: I write to database a document
- When: I read the document
- Then: the result is the expected document
Test class B:
- Given: Repo available
- When: I write a document
- Then: it writes without exceptions
Now imagine that both test classes are run in parallel, sometimes the following problems arise:
- Test A executes and writes a document with Id 1. At the same time Test B tries to write a document with Id 1 in its
when
and it fails because there is a document already in the same database with the same Id. - Test B executes, and it has a teardown/cleanup that deletes the document at the end of its test. At the same time the Test A was about to read a document that expects to be there... and it fails because the document has been deleted (from the Test B)
The question is: Is it even possible to run integration tests in parallel and achieve idempotent Given
without facing problems because one test messes up with the data of the other test?
I thought of a few ideas but I have no experience with it, so I am looking for opinions and a solution.
- Solution A: Ensuring every test class uses data that no other test accesses. For example by providing different Ids to the test data. This could indeed solve the problem but it forces developers to be aware of which Ids are being used by other tests.
- Solution B: Have some kind of assembly Given and Teardown that prepares the scenario for every test. Again, we would rely on something greater than the test class itself and it seems it violates the GivenThenWhen philosophy I want to follow.
xUnit has the possibility of different shared context between tests but I don't see how that fits with my template either: https://xunit.github.io/docs/shared-context.
How do you handle those integration test scenarios with xUnit? Ta
UPDATE 1: Here is an example of how I create my tests using the GTW philosophy and xUnit. These facts sometimes fail because they cannot insert a document with an Id that already exists (because other test classes that use a document with the same id are running at the same time and didn't cleanup yet)
public static class GetAllTests
{
public class Given_A_Valid_Filter_When_Getting_All
: Given_WhenAsync_Then_Test
{
private ReadRepository<FakeDocument> _sut;
private Exception _exception;
private Expression<Func<FakeDocument, bool>> _filter;
private IEnumerable<FakeDocument> _result;
private IEnumerable<FakeDocument> _expectedDocuments;
protected override void Given()
{
_filter = x => true;
var cursorServiceMock = new Mock<ICursorService<FakeDocument>>();
var all = Enumerable.Empty<FakeDocument>().ToList();
cursorServiceMock
.Setup(x => x.GetList(It.IsAny<IAsyncCursor<FakeDocument>>()))
.ReturnsAsync(all);
var cursorService = cursorServiceMock.Object;
var documentsMock = new Mock<IMongoCollection<FakeDocument>>();
documentsMock
.Setup(x => x.FindAsync(It.IsAny<Expression<Func<FakeDocument, bool>>>(),
It.IsAny<FindOptions<FakeDocument, FakeDocument>>(), It.IsAny<CancellationToken>()))
.ReturnsAsync(default(IAsyncCursor<FakeDocument>));
var documents = documentsMock.Object;
_sut = new ReadRepository<FakeDocument>(documents, cursorService);
_expectedDocuments = all;
}
protected override async Task WhenAsync()
{
try
{
_result = await _sut.GetAll(_filter);
}
catch (Exception exception)
{
_exception = exception;
}
}
[Fact]
public void Then_It_Should_Execute_Without_Exceptions()
{
_exception.Should().BeNull();
}
[Fact]
public void Then_It_Should_Return_The_Expected_Documents()
{
_result.Should().AllBeEquivalentTo(_expectedDocuments);
}
}
public class Given_A_Null_Filter_When_Getting_All
: Given_WhenAsync_Then_Test
{
private ReadRepository<FakeDocument> _sut;
private ArgumentNullException _exception;
private Expression<Func<FakeDocument, bool>> _filter;
protected override void Given()
{
_filter = default;
var cursorService = Mock.Of<ICursorService<FakeDocument>>();
var documents = Mock.Of<IMongoCollection<FakeDocument>>();
_sut = new ReadRepository<FakeDocument>(documents, cursorService);
}
protected override async Task WhenAsync()
{
try
{
await _sut.GetAll(_filter);
}
catch (ArgumentNullException exception)
{
_exception = exception;
}
}
[Fact]
public void Then_It_Should_Throw_A_ArgumentNullException()
{
_exception.Should().NotBeNull();
}
}
}
UPDATE 2: If I make random Ids, sometimes I will also get into problems because the expected documents to be retrieve from DB contain more items than the ones the test expects (because, again, other tests running in parallel have written more documents in the database).