How do I enforce exception message with ExpectedException attribute

Question

I thought these two tests should behave identically, in fact I have written the test in my project using MS Test only to find out now that it does not respect the expected message in the same way that NUnit does.

NUnit (fails):

[Test, ExpectedException(typeof(System.FormatException), ExpectedMessage = "blah")]
public void Validate()
{
    int.Parse("dfd");
}

MS Test (passes):

[TestMethod, ExpectedException(typeof(System.FormatException), "blah")]
public void Validate()
{
    int.Parse("dfd");
}

No matter what message I give the ms test, it will pass.

Is there any way to get the ms test to fail if the message is not right? Can I even create my own exception attribute? I would rather not have to write a try catch block for every test where this occurs.

score 31 · Answer 1 · edited Feb 10 '20 at 23:31

We use this attribute all over the place and we clearly misunderstood the second parameter (shame on us).

However, we definitely have used it to check the exception message. The following was what we used with hints from this page. It doesn't handle globalization, or inherited exception types, but it does what we need. Again, the goal was to simply RR 'ExpectedException' and swap it out with this class. (Bummer ExpectedException is sealed.)

public class ExpectedExceptionWithMessageAttribute : ExpectedExceptionBaseAttribute
{
    public Type ExceptionType { get; set; }

    public string ExpectedMessage { get; set; }

    public ExpectedExceptionWithMessageAttribute(Type exceptionType)
    {
        this.ExceptionType = exceptionType;
    }

    public ExpectedExceptionWithMessageAttribute(Type exceptionType, string expectedMessage)
    {
        this.ExceptionType = exceptionType;
        this.ExpectedMessage = expectedMessage;
    }

    protected override void Verify(Exception e)
    {
        if (e.GetType() != this.ExceptionType)
        {
            Assert.Fail($"ExpectedExceptionWithMessageAttribute failed. Expected exception type: {this.ExceptionType.FullName}. " +
                $"Actual exception type: {e.GetType().FullName}. Exception message: {e.Message}");
        }

        var actualMessage = e.Message.Trim();
        if (this.ExpectedMessage != null)
        {
            Assert.AreEqual(this.ExpectedMessage, actualMessage);
        }

        Debug.WriteLine($"ExpectedExceptionWithMessageAttribute:{actualMessage}");
    }
}

I tried to achieve the same thing using PostSharp, but inheriting the existing attribute makes alot more sense! — RJ Lohan, Sep 25 '13 at 22:54
Literally just made this class myself. Though I called it `PrintExpectedExceptionAttribute` :) — dav_i, Jan 28 '14 at 14:01
Nice! As they say, 'Great minds think alike and do exactly the same work in parallel'. I gotta say, we use this attribute all the time still. — BlackjacketMack, Jan 28 '14 at 14:47
Thought I'd add that we now basically use a ProjectAssert (with 'Project' being our Project name) that has a static method: ExpectException(Action action,Type exceptionType, string message = null); So it could look like ProjectAssert.ExpectException(()=>{var x = 0; var y = 1/x;},typeof(DivideByZeroException)). It's nice getting it inline but basically does the exact same thing the attribute above does. — BlackjacketMack, Feb 03 '16 at 11:19

score 23 · Accepted Answer · answered Jan 17 '11 at 05:39

23

That mstest second parameter is a message that is printed out when the test fails. The mstest will succeed if a formatexception is thrown. I found this post that may be useful

http://blogs.msdn.com/b/csell/archive/2006/01/13/expectedexception-might-not-be-what-you-ve-expected.aspx

answered Jan 17 '11 at 05:39

rcravens

8,320
2
33
26

2

Thanks, thats some great information. Do you know a way using ms test to write a test that behaves the same as the nunit one? – CRice Jan 17 '11 at 05:50
1

I might look at creating a custom attribute like the nunit one. – CRice Jan 17 '11 at 05:51
9

The link is dead. – Thibault D. Mar 19 '19 at 09:21
the link is dead – Jurion Jan 01 '23 at 21:23

score 16 · Answer 3 · edited May 27 '21 at 06:39

16

@rcravens is correct - the second param is a message that is printed if the test fails. What I've done to work around this is craft my tests a little differently. Admittedly, I don't love this approach, but it works.

[TestMethod]
public void Validate()
{
    try
    {
        int.Parse("dfd");
    
        // Test fails if it makes it this far
        Assert.Fail("Expected exception was not thrown.");
    }
    catch (ArgumentNullException ex) // <-- Expected Exception class
    {
        Assert.AreEqual("blah", ex.Message);
    }
    catch (Exception ex) // All the other exceptions
    {
        Assert.Fail("Thrown exception was of the wrong type");
    }
}

edited May 27 '21 at 06:39

Draco Ater

20,820
8
62
86

answered May 16 '12 at 13:57

Jeremy Wiggins

7,239
6
41
56

I am a fan of this approach now if using mstest – CRice Oct 05 '15 at 23:03
Thanks, this will quickly do what I want it to do, validate my message – Malcolm Anderson Apr 29 '16 at 22:17
This is actually pretty crafty workaround.. I love it! – Piotr Kula Nov 23 '17 at 13:34
3

This would validate ONLY the message. If you also want to validate the exception type, the catch would have to be something like catch (ArgumentNullException ex){ Assert.AreEqual("blah", ex.Message); } catch (Exception ex){ Assert.Fail("Thrown exception was of the wrong type"); } – bkqc Dec 11 '18 at 15:47

score 9 · Answer 4 · answered Sep 20 '13 at 18:35

I extended answer from BlackjacketMack for our project by adding support for contains, case-insensitive and ResourcesType-ResourceName combinations.

Usage example:

public class Foo
{
    public void Bar(string mode)
    {
        if (string.IsNullOrEmpty(mode)) throw new InvalidOperationException(Resources.InvalidModeSpecified);
    }

    public void Bar(int port)
    {
        if (port < 0 || port > Int16.MaxValue) throw new ArgumentOutOfRangeException("port");
    }
}

[TestClass]
class ExampleClass
{
    [TestMethod]
    [ExpectedExceptionWithMessage(typeof(InvalidOperationException), typeof(Samples.Resources), "InvalidModeSpecified")]
    public void Raise_Exception_With_Message_Resource()
    {
        new Foo().Bar(null);
    }

    [TestMethod]
    [ExpectedExceptionWithMessage(typeof(ArgumentOutOfRangeException), "port", true)]
    public void Raise_Exeception_Containing_String()
    {
        new Foo().Bar(-123);
    }
}

Here is the updated class:

public class ExpectedExceptionWithMessageAttribute : ExpectedExceptionBaseAttribute
{
    public Type ExceptionType { get; set; }

    public Type ResourcesType { get; set; }

    public string ResourceName { get; set; }

    public string ExpectedMessage { get; set; }

    public bool Containing { get; set; }

    public bool IgnoreCase { get; set; }

    public ExpectedExceptionWithMessageAttribute(Type exceptionType)
        : this(exceptionType, null)
    {
    }

    public ExpectedExceptionWithMessageAttribute(Type exceptionType, string expectedMessage)
        : this(exceptionType, expectedMessage, false)
    {
    }

    public ExpectedExceptionWithMessageAttribute(Type exceptionType, string expectedMessage, bool containing)
    {
        this.ExceptionType = exceptionType;
        this.ExpectedMessage = expectedMessage;
        this.Containing = containing;
    }

    public ExpectedExceptionWithMessageAttribute(Type exceptionType, Type resourcesType, string resourceName)
        : this(exceptionType, resourcesType, resourceName, false)
    {
    }

    public ExpectedExceptionWithMessageAttribute(Type exceptionType, Type resourcesType, string resourceName, bool containing)
    {
        this.ExceptionType = exceptionType;
        this.ExpectedMessage = ExpectedMessage;
        this.ResourcesType = resourcesType;
        this.ResourceName = resourceName;
        this.Containing = containing;
    }

    protected override void Verify(Exception e)
    {
        if (e.GetType() != this.ExceptionType)
        {
            Assert.Fail(String.Format(
                            "ExpectedExceptionWithMessageAttribute failed. Expected exception type: <{0}>. Actual exception type: <{1}>. Exception message: <{2}>",
                            this.ExceptionType.FullName,
                            e.GetType().FullName,
                            e.Message
                            )
                        );
        }

        var actualMessage = e.Message.Trim();

        var expectedMessage = this.ExpectedMessage;

        if (expectedMessage == null)
        {
            if (this.ResourcesType != null && this.ResourceName != null)
            {
                PropertyInfo resourceProperty = this.ResourcesType.GetProperty(this.ResourceName, BindingFlags.Static | BindingFlags.NonPublic | BindingFlags.Public);
                if (resourceProperty != null)
                {
                    string resourceValue = null;

                    try
                    {
                        resourceValue = resourceProperty.GetMethod.Invoke(null, null) as string;
                    }
                    finally
                    {
                        if (resourceValue != null)
                        {
                            expectedMessage = resourceValue;
                        }
                        else
                        {
                            Assert.Fail("ExpectedExceptionWithMessageAttribute failed. Could not get resource value. ResourceName: <{0}> ResourcesType<{1}>.",
                            this.ResourceName,
                            this.ResourcesType.FullName);
                        }
                    }
                }
                else
                {
                    Assert.Fail("ExpectedExceptionWithMessageAttribute failed. Could not find static resource property on resources type. ResourceName: <{0}> ResourcesType<{1}>.",
                        this.ResourceName,
                        this.ResourcesType.FullName);
                }
            }
            else
            {
                Assert.Fail("ExpectedExceptionWithMessageAttribute failed. Both ResourcesType and ResourceName must be specified.");
            }
        }

        if (expectedMessage != null)
        {
            StringComparison stringComparison = this.IgnoreCase ? StringComparison.OrdinalIgnoreCase : StringComparison.Ordinal;
            if (this.Containing)
            {
                if (actualMessage == null || actualMessage.IndexOf(expectedMessage, stringComparison) == -1)
                {
                    Assert.Fail(String.Format(
                                    "ExpectedExceptionWithMessageAttribute failed. Expected message: <{0}>. Actual message: <{1}>. Exception type: <{2}>",
                                    expectedMessage,
                                    e.Message,
                                    e.GetType().FullName
                                    )
                                );
                }
            }
            else
            {
                if (!string.Equals(expectedMessage, actualMessage, stringComparison))
                {
                    Assert.Fail(String.Format(
                                    "ExpectedExceptionWithMessageAttribute failed. Expected message to contain: <{0}>. Actual message: <{1}>. Exception type: <{2}>",
                                    expectedMessage,
                                    e.Message,
                                    e.GetType().FullName
                                    )
                                );
                }
            }
        }
    }
}

As explained here https://stackoverflow.com/a/53730206/955444, this does not work as expected in version 11 (VS2012) because it will always show "Exception has been thrown by the target of an invocation". It is fixed at least starting version 14. — bkqc, Dec 11 '18 at 18:30

CRice · Answer 5 · 2011-01-18T01:29:52.413

4

You can use code from this project explained in this article to create an ExpectedExceptionMessage attribute which will work with MS Test to get the desired result.

ms test (fails):

[TestClass]
public class Tests : MsTestExtensionsTestFixture
{
    [TestMethod, ExpectedExceptionMessage(typeof(System.FormatException), "blah")]
    public void Validate()
    {
        int.Parse("dfd");
    }
}

edited Jan 18 '11 at 01:29

answered Jan 18 '11 at 01:24

CRice

12,279
7
57
84

score 1 · Answer 6 · answered Aug 29 '13 at 03:02

1

I wrote this a while back, so maybe there is a better way in VS2012.

http://dripcode.blogspot.com.au/search?q=exception

answered Aug 29 '13 at 03:02

John Roth

11
1

vladimir · Answer 7 · 2021-04-28T18:09:33.347

1

Try this modern way:

[TestMethod]
public async Task DoSmthAsync_WithWrongParam_ThrowHttpRequestException()
{
    var smbdTask = service.DoSmthAsync(/* call params */);

    var exception = await Assert.ThrowsExceptionAsync<HttpRequestException>(() => smbdTask);
    Assert.AreEqual(HttpStatusCode.NotFound, exception.StatusCode);
}

edited Apr 28 '21 at 18:09

answered Apr 28 '21 at 17:50

vladimir

13,428
2
44
70

Yes this question is ancient now. These days I use fluent assertions in the test method code and no attributes similar to your answer. – CRice Apr 28 '21 at 22:43

score 0 · Answer 8 · edited Apr 17 '13 at 14:13

Try This. In This example I have a greeting message that displays the greeting message for a given name. When the name parameter is empty or null system throws an exception with message. The ExceptionAssert.Throws verify both in MsTest.

[TestMethod]
public void Does_Application_Display_Correct_Exception_Message_For_Empty_String()
{
    // Arrange
    var oHelloWorld = new HelloWorld();

    // Act

    // Asset

    ExceptionAssert.Throws<ArgumentException>(() => 
            oHelloWorld.GreetingMessge(""),"Invalid Name, Name can't be empty");
}

[TestMethod]
public void Does_Application_Display_Correct_Exception_Message_For_Null_String()
{
    // Arrange
    var oHelloWorld = new HelloWorld();

    // Act

    // Asset

    ExceptionAssert.Throws<ArgumentNullException>(() => 
        oHelloWorld.GreetingMessge(null), "Invalid Name, Name can't be null");
}

ExceptionAssert doesn't seems to be bundled in Microsoft UnitTesting, are you using a third party plugin? — ForceMagic, Jul 26 '13 at 20:33

How do I enforce exception message with ExpectedException attribute

8 Answers8