103

A bit confused on the differences of Mock frameworks like NMock vs the VS 2011 Fakes Framework. Going through MSDN, what I understand is that Fakes allow you to mock your dependencies just like RhinoMock or NMock, however the approach is different, Fakes generates code to achive this functionality but Mocks framework does not. So is my understanding correct? Is Fakes just another Mock framework

Peter
  • 27,590
  • 8
  • 64
  • 84
Nairooz NIlafdeen
  • 1,150
  • 2
  • 9
  • 13

5 Answers5

195

Your question was about how the MS Fakes framework is different from NMock and it appears the other answers have resolved some of that, but here is some more information regarding how they are the same and how they are different. NMock is also similar to RhinoMocks and Moq, so I'm grouping them in with NMock.

There are 3 major differences I see right off between NMock/RhinoMocks/Moq and the MS Fakes Framework:

  • The MS fakes framework uses generated code, much like Accessors in prior versions of Visual Studio instead of generic types. When you want to use the fakes framework for a dependency, you add the assembly that contains the dependency to the references of the test project and then right-click on it to generate the test doubles (stubs or shims). Then when you are testing, you are actually using these generated classes instead. NMock uses generics to accomplish the same thing (i.e. IStudentRepository studentRepository = mocks.NewMock<IStudentRepository>()). In my opinion, the MS Fakes framework approach inhibits code navigation and refactoring from within the tests since you are actually working against a generated class, not your real interface.

  • The MS fakes framework supplies stubs and moles (shims), whereas NMock, RhinoMocks, and Moq all provide stubs and mocks. I really don't understand MS's decision to not include mocks and I am, personally not a fan of moles for reasons described below.

  • With the MS fakes framework, you supply an alternative implementation of the methods you want to stub. Within these alternate implementations, you can specify the return values and track information about how or if the method was called. With NMock, RhinoMocks and Moq, you generate a mock object and then use that object to specify stubbed return values or to track interactions (whether and how the methods were called). I find the MS fakes approach more complex and less expressive.

To clarify the difference in what the frameworks provide: NMock, RhinoMocks and Moq all provide two types of test doubles (stubs and mocks). The fakes framework provides stubs and moles (they call them shims), and unfortunately does not include mocks. In order to understand the differences and similarities between NMock and MS Fakes, it is helpful to understand what these different types of test doubles are:

Stubs: Stubs are used when you need to provide a values for methods or properties that will be asked of your test doubles by the method under test. For example, when my method under test calls the DoesStudentExist() method of the IStudentRepository test double, I want it to return true.

The idea of stubs in NMock and MS fakes is the same, but with NMock you would do something like this:

Stub.On(mockStudentRepository).Method("DoesStudentExist").Will(Return.Value(true));

And with MSFakes you would do somethign like this:

IStudentRepository studentRepository = new DataAccess.Fakes.StubIStudentRepository() // Generated by Fakes.
{
    DoesStudentExistInt32 = (studentId) => { return new Student(); }
};

Notice in the MS Fakes example you create an entirely new implementation for the DoesStudentExist method (Note that it is called DoesStudentExistInt32 because the fakes framework appends the parameter data types to the method names when it generates the stub objects, I think this obscures the clarity of the tests). To be honest the NMock implementation also bugs me because it uses a string to identify the method name. (Forgive me if I've misunderstood how NMock is intended to be used.) This approach really inhibits refactoring and I'd highly recommend RhinoMocks or Moq over NMock for this reason.

Mocks: Mocks are used to verify interaction between your method under test and its dependencies. With NMock, you do this by setting expectations similar to this:

Expect.Once.On(mockStudentRepository).Method("Find").With(123);

This is another reason why I'd prefer RhinoMocks and Moq over NMock, NMock uses the older expectation style whereas RhinoMocks and Moq both support the Arrange/Act/Assert approach where you specify you expected interactions as assertions at the end of the test like this:

stubStudentRepository.AssertWasCalled( x => x.Find(123));

Again, note that RhinoMocks uses a lambda instead of a string to identify the method. The ms fakes framework does not provide mocks at all. This means that in your stubbed out implementations (see the description of stubs above) you have to set variables that you later verify were set correctly. That would look something like this:

bool wasFindCalled = false;

IStudentRepository studentRepository = new DataAccess.Fakes.StubIStudentRepository() 
{
    DoesStudentExistInt32 = (studentId) => 
        { 
            wasFindCalled = true;
            return new Student(); 
        }
};

classUnderTest.MethodUnderTest();

Assert.IsTrue(wasFindCalled);

I find this approach to be a little convoluted since you have to track the call up in the stub and then assert it later in the test. I find the NMock, and especially the RhinoMocks, examples to be more expressive.

Moles (Shims): To be frank, I do not like moles, because of their potential for misuse. One of the things I like so much about unit testing (and TDD in particular) is that testing your code helps you to understand where you have written poor code. This is because testing poorly written code is difficult. This is not true when using moles because moles are actually designed to allow you to test against dependencies that are not injected or to test private methods. They work similarly to stubs, except that you use a ShimsContext like this:

using (ShimsContext.Create())
{
    System.Fakes.ShimDateTime.NowGet = () => { return new DateTime(fixedYear, 1, 1); };
}

My worry with shims is that people will start seeing them as "an easier way to unit test" because it doesn't force you to write code the way you should. For a more complete write-up on this concept see this post of mine:

For more information on some concerns related to the fakes frameworks take a look at these posts:

If you're interested in learning RhinoMocks here's a Pluralsight training video (full disclosure - I wrote this course and get paid royalties for views, but I think it applies to this discussion so I'm including it here):

Amittai Shapira
  • 3,749
  • 1
  • 30
  • 54
Jim Cooper
  • 5,113
  • 5
  • 30
  • 35
  • 15
    @Jim I disagree that allowing one to "test against dependencies that are not injected" is a bad thing. On the contrary, in practice, this ability helps to avoid cluttering up a codebase with lots of pointless interfaces and the associated "object wiring" configuration/complexity. I have seen a few projects already (both Java and .NET) which had hundreds of Java/C# interfaces with only a single implementing class, and lots of useless XML configuration (for Spring DI beans, or in Web.config for the MS Unity framework). Such dependencies are better *not* injected. – Rogério Jan 16 '13 at 14:37
  • 20
    @Rogerio, I have worked on multiple projects with thousands of classes that follow this model and when dependency injection is done right (if you're using XML configuration you're not doing it right, imho), it is simple and relatively painless. On the other hand, I've worked on systems that don't do this and the coupling between classes has always caused me significant pain. Testing is not the reason for using dependency injection (although it's not a bad reason). The real reason is the loosened coupling. Having interfaces that are implemented by a single class has never caused me pain. – Jim Cooper Feb 01 '13 at 19:02
  • 19
    @Jim, I see the ability to test bad design as a great plus. The first thing you want to do before refactoring legacy code towards a better design, is write tests. – Thomas Materna Mar 28 '13 at 13:27
  • I think Fake framework's `Observer` serves the purpose of Mock somehow. – hardywang Jul 09 '13 at 23:55
  • Thanks for a great overview of the alternatives to MS Test. Sad to see another potentially standard free framework (Moles) turned into a premium product. Also for your detailed examples and point of caution with NMock. – Tony Wall May 01 '14 at 12:04
  • Sometimes Fakes's moles are the only way to fully test (against a system out of your control). – Cameron Verhelst May 19 '14 at 06:34
  • 4
    My rule is I use Fakes only when I have to Shim a shared/static method or a class that I don't have access to implement an interface. For everything else I use Moq. Fakes are slower (because the fake assemblies need to be generated), and it takes more coding effort to match the functionality that you get with Moq. – Nick Sep 23 '14 at 13:23
  • @JimCooper Given this answer of yurs, do you support interfacing a TimeProvider and replacing all DateTime.Now calls in the system to call to that provider? this doesn't sound like a great code improvement apart from the testability. And it is no coincidence imo that this particular example is chosen as a good use for the Shims functionality of MS Fakes. Also, with your answer being the most complete here, i don't think there are ANY frameworks that do exactly what MS Fakes does with Shims. – zaitsman Dec 29 '14 at 05:35
  • @zaitsman We actually do use something like that in our code where I work. It is called UtcTime and has methods on it like UtcTime.Now() and UtcTime.Today(), etc. It could certainly be expanded to have more domain-specific functions. We use this universally throughout our code. – Jim Cooper Dec 29 '14 at 23:30
  • @JimCooper yup i am familiar with the concept. I just wanted to see what benefits would you attribute to this approach apart from the ability to unit test this? – zaitsman Dec 30 '14 at 20:24
  • 1
    @zaitsman, not much value beyond testing, but it's also only a few lines of code. I'm sure it's a style difference and if you prefer to shim rather than write a DateTime wrapper class, more power to you. I'm not suggesting that there are no good uses for shims, but I am not likely to use them personally. The fact that MS included shims and not mocks, however, I think is very unfortunate and will lead to some bad practices by some people who are new to testing. – Jim Cooper Jan 02 '15 at 18:06
  • 1
    @zaitsman I *do* provide an IDateTimeNow service in all my "non throwaway" projects, just for that reason .. – user2864740 Jan 06 '15 at 17:55
  • 6
    @CarloV.Dango First of all, I know perfectly well that Dependency Injection does *not* require separate interfaces; my previous comment was just alluding to the tendency to create such interfaces when using DI. Secondly, "IOC" (Inversion of Control) has *nothing to do* with DI! (The first is about inversion of the *flow of control* between methods, while the second is about the resolution of dependencies for a component/object.) By confusing IOC and DI, it's you that's showing ignorance, not me; and, frankly, this site does not deserve people insulting others, so please, lets keep it civil. – Rogério Apr 09 '15 at 14:52
  • 1
    The links are broken. At the moment they appear to be at http://blog.pluralsight.com/vs11-fakes-framework, http://blog.pluralsight.com/vs11-fakes-framework-harmful-part-two and http://blog.pluralsight.com/solid-code-for-solid-reasons – Harry Biscuit Apr 27 '15 at 15:20
  • @JimCooper if a method is not using any instance property, it is better to convert it to static, I don't think using static method and classes is antipattern. Also, I don't understand why there is a need to mock the static methods and private methods, Calculator.Add(2,2) should return 4 only, mocking it to return 5 doesn't make sense to me. Also static methods and classes are automatically provided coverage, you won't need to write test class for them. – Deepak Mishra Apr 29 '22 at 08:53
25

You are correct, but there's more to the story. The most important things to take away from this answer are:

  1. Your architecture should make proper use of stubs and dependency injection, rather than relying on the crutch of Fakes and mocks

  2. Fakes and mocks are useful for testing code you shouldn't or can't change, such as:

    • Legacy code that does not make use (or efficient use) of stubs
    • 3rd party APIs
    • Resources for which you have no source code

Shims (known as "Moles", during development) is indeed a mocking framework that operates by way of detouring calls. Instead of painstakingly building a mock (yes, even using Moq is relatively painful!), shims simply use the production code object already in place. Shims simply re-route the call from the production target to the test delegate.

Stubs are generated from interfaces in the target project. The Stub object is just that -- an implementation of the interface. The benefit to using the Stub type is that you can quickly generate a stub without cluttering up your test project with many one-time use stubs, not to mention wasting time creating them. Of course, you should still create concrete stubs, for use across many tests.

Efficiently implementing Fakes (Shims, Mocks and Stub types) takes a little getting used to, but is well worth the effort. I have personally saved weeks of development time, through use of the Shims/Mole, Mocks, and Stub types. I hope you have as much fun with the technology as I have!

Snesh
  • 555
  • 1
  • 10
  • 20
Mike Christian
  • 1,526
  • 1
  • 15
  • 27
  • 12
    Downvoted based on lack of specifics and some terminology issues with your comments on Fakes. Fakes is a general term for all types of test doubles and also the name MS used for their fakes library. In their library only Shims are actually Moles, stubs are not. Regarding the need for examples, I would like to see an example in your answer that shows how creating a shim (or stub) with Fakes is more simple than using Moq. If you make changes to your answer to resolve these, I'll change my downvote. – Jim Cooper Dec 04 '12 at 19:41
  • 1
    But does add good justification for use of shims (moles), i.e. 3rd party code, system/SDK APIs. It's not just about when you are working with your own in-house solutions. So I'll up you one vote to even that out :-) – Tony Wall May 01 '14 at 12:26
  • 2
    @Mike and Jim.. I did effort to correct terminology used in this answer. Please let me know if we can make it better. Thanks. – Snesh Dec 19 '14 at 09:29
15

As I understand it, the Visual Studio team wanted to avoid competing with the various mock libraries available for .NET. MS often faces hard decisions like this. They are dammed if they don't provide certain functionality ("why doesn't MS provide us with a mock library; mocks are such a common requirement?") and damned if they do ("why is Microsoft acting so aggressively and driving its natural supporters out of the market?") Very often, but not always, they decide to hold back from simply providing their own alternative to available and well-received technologies. That seems to be the case here.

The shim feature of Fakes is really, really useful. Sure, there are dangers. It takes some discipline to ensure you only use this where necessary. However, it fills a big gap. My main complaint is that it is only delivered with the Ultimate edition of VS 2012 and therefore will only be available to a subsection of the .NET development community. What a pity.

Charles Young
  • 159
  • 1
  • 3
13

Fakes includes two different kinds of "fake" object. The first, called a "stub", is essentially an auto-generated dummy whose default behaviour can (and usually would) be overridden to make it a more interesting mock. It does, however, lack some of the features that most of the currently available mocking frameworks offer. For example, if you want to check that a method on a stub instance was invoked, you would need to add the logic for this yourself. Basically, if you're authoring your own mocks manually now, stubs would probably seem like an improvement. However, if you're already using a more full-featured mocking framework, you might feel like there are some important pieces missing from Fakes stubs.

The other category of object offered by Fakes, called a "shim", exposes a mechanism for replacing behaviour of dependencies that have not been (or cannot be) decoupled adequately for standard replacement via mocks. AFAIK, TypeMock is the only one of the major mocking frameworks that currently offers this sort of functionality.

BTW, if you have tried out Moles before, Fakes is essentially the same thing, finally making its way out of Microsoft Research and into an actual product.

Nicole Calinoiu
  • 20,843
  • 2
  • 44
  • 49
  • 1
    Update: Moles has now been integrated into MS Fakes. "The Fakes Framework in Visual Studio 2012 is the next generation of Moles & Stubs. Fakes is different from Moles, however, so moving from Moles to Fakes will require some modifications to your code. The Moles framework will not be supported in Visual Studio 2012." Source: http://research.microsoft.com/en-us/projects/moles/ – Sire Jul 03 '14 at 14:02
1

Regarding Fake (Shim + Stub) objects, it has been well defined above, though I guess the last paragraph in the last comment summarizes the whole situation pretty well.

Though a lot of people will argue that Fake (Shim + Stub) objects are good assets to have in some unit testing cases, the downside is that no matter if you're using Visual Studio 2012 or Visual Studio 2013, these options are ONLY available with Premium or Ultimate versions. IOW, this means that you WILL NOT be to run ANY of those Fakes (Shim + Stub) on any Pro version.

You may probably see the Fakes (Shim + Stub) menu option on Pro versions, but beware, there are some pretty strong chances that you will end up with absolutely nothing... It won't generate any compilation error telling you that something important is missing, options are just not there, so don't waste your time...

It's an important factor to consider in a dev team, especially if one is the only one using Ultimate version while everybody else uses Pro version... Moq on the other hand can easily be installed through Nuget no matter which Visual Studio version you use. I had no problem using Moq, the key with any tool is to know what they're used for and how to use them properly ;)

Matteo Italia
  • 123,740
  • 17
  • 206
  • 299
SirXamelot
  • 11
  • 1
  • 3
  • 1
    Please format your question into smaller paragraphs so that it's easier to read. –  Jul 06 '14 at 00:29
  • @SirXamelot, not refactoring the code you can't change the output of .net provided static calls e.g. DateTime.Now or Guid.NewGuid – zaitsman Dec 29 '14 at 05:27
  • exactly, I raised several times to MS blogs that keeping the MS.Fakes only in entreprise edition prevent them to become a mainsteam product. As you said, Moq works just fine with Pro edition and test pipeline in Gitlab. – EricBDev Aug 12 '21 at 11:36