Under what circumstances might DateTime.Today return yesterday's date?

Question

In our codebase there is an XUnit theory that uses the attribute [Theory, MemberData(nameof(CancelledDates))]. Here is the method that provides the MemberData.

public static IEnumerable<object[]> CancelledDates()
{
    return new[]
    {
        new object[] { DateTime.Today.AddDays(5), DateTime.Today.AddDays(10), false, "Incident date cannot be before the Cover Start date of the Policy." },
        new object[] { DateTime.Today.AddDays(-10), DateTime.Today.AddDays(-5), false, "Incident date cannot be after the Policy Cover End date." },
        new object[] { DateTime.Today.AddDays(-5), DateTime.Today, true, string.Empty },
        new object[] { DateTime.Today, DateTime.Today.AddDays(5), true, string.Empty },
    };
}

The problem is the third entry; when I ran tests around 16:00 on 2023/07/13 the test failed because the dates were one day earlier than they should have been. Rather than 2023-07-08T00:00:00.0000000+02:00 and 2023-07-13T00:00:00.0000000+02:00 as expected, the dates provided to the test were 2023-07-07T00:00:00.0000000+02:00 and 2023-07-12T00:00:00.0000000+02:00. This results in the test failing as the second date is expected not to be earlier than today in the code under test.

The strange thing is that this does not happen every time the test is run. It seems to happen only when running tests in Visual Studio and Azure DevOps builds that require tests to pass never seem to fail on this test. The date and time on my machine are correct and the time zone is set to UTC+2.

This has been happening to me randomly for several months; I'm tired of it and would love to find a solution or at least understand why it happens. Any ideas?

I suspect there's insufficient data to give a valid answer. Maybe the third parameter (bool) is affecting the values of first two parameters (DateTime). — Dialecticus, Jul 13 '23 at 14:55
Are you sure the dates passed to the other tests are correct, and that the tests are not succeeding with false positives? — DisplayName, Jul 13 '23 at 16:02
I've encountered an issue before where tests in VS that depend on dynamic data like yours fail because the data has to be predetermined in order to "name" the test in the test window. I believe this happens at build. You say it happens randomly, I suspect it happens after a non-clean test run with a build from the previous day. Clean your build and the test names (and therefore data) will be reset. — DisplayName, Jul 13 '23 at 16:07
One solution is to [mock `DateTime.Today`](https://stackoverflow.com/questions/2425721/unit-testing-datetime-now) and used fixed values in your tests. — John Wu, Jul 13 '23 at 20:00

DisplayName · Accepted Answer · 2023-07-13T17:11:28.347

I believe your issue is that xUnit serialises tests at build time in VS in order to generate names for the test runner: https://github.com/xunit/xunit/issues/1473#issuecomment-333226539

When you later run the tests without a clean build (perhaps the next day), the dynamic data is stale. This is not an issue in DevOps as it builds, serialises and runs the tests in one go.

I had a look to find a fix, and looked back at some of my own changes as I have encountered this before, but cannot find anything satisfactory. I have a vague memory there is a way to force xUnit or the VS test runner to display the entire Theory as one test with the (static) method name, but this isn't much of a "fix".

The best solution in your instance is probably to pass the offsets into the test rather than the dates themselves, meaning you'd provide static data to the tests and VS/xUnit can generate test names without locking in stale data:

public static IEnumerable<object[]> CancelledDates()
{
    return new[]
    {
        new object[] { 5, 10, false, "Incident date cannot be before the Cover Start date of the Policy." },
        new object[] { -10, -5, false, "Incident date cannot be after the Policy Cover End date." },
        new object[] { -5, 0, true, string.Empty },
        new object[] { 0, 5, true, string.Empty },
    };
}

Then, inside your test method you can simply parse the offsets with DateTime.Today.AddDays(offset).

As a general rule of thumb, don't pass any data via MemberData that you wouldn't be able to pass via InlineData - i.e. non-constant data.

Hope this helps :)

Take this answer with a pinch of salt as it's mostly from distant memory, but the general theory and the solution should hold. — DisplayName, Jul 13 '23 at 16:58
Thanks for this explanation. It certainly makes sense now I know that XUnit is serializing the theory cases at build time. I have refactored to have startOffsetDays and endOffsetDays coming in to the test and then do the start and end date calculation in the test. — Steve Crane, Jul 14 '23 at 07:28

Under what circumstances might DateTime.Today return yesterday's date?

1 Answers1