1

You can skip to my approach if you don't mind what I'm actually trying to do.

What I'm trying to do
Hey I'm trying to make mutant testing,
inspired by the talk
http://www.infoq.com/presentations/kill-better-test

But I'm using c# and the best mutant libraries are made in java such as
http://pitest.org/

There are some frameworks for c# such as ninjaturtles and visualmutator, but they both doesn't work in my computer for some reason(I get a weird error). and also I thought it would be interesting creating my own.

About mutant testing
For those who doesn't know what is mutant testing, it's a testing for the tests, most people use code coverage to check that their test cover all the scenarios, but it's not enough, cause just because it gets to a piece of code doesn't mean it tests it. It changes a piece of code, and if the tests still pass it means you didn't tested the piece of code.

My approach
So I've tried starting with a simple code that gets the il codes of a method.

        var classType = typeof(MethodClass);
        var methodInfo = classType.GetMethod("ExecuteMethod", BindingFlags.NonPublic | BindingFlags.Static);

        byte[] ilCodes = methodInfo.GetMethodBody().GetILAsByteArray();

this is the MethodClass I'm trying to change:

public class MethodClass
{
    private static int ExecuteMethod()
    {
        var i = 0;
        i += 5;
        if (i >= 5)
        {
            i = 2;
        }
        return i;
    }
}

now I'm trying to replace the ils

        for (int i = 0; i < ilCodes.Length; i++)
        {
            if (ilCodes[i] == OpCodes.Add.Value)
            {
                ilCodes[i] = (byte)OpCodes.Sub.Value;
            }
        }

but then I'm not sure how to update my function to work with the new il codes.

I've tried using 

        var dynamicFunction = new DynamicMethod("newmethod", typeof(int), null);
        var ilGenerator = dynamicFunction.GetILGenerator();

and then the il generator has a function emit, that gets operator and value, so I could use this. but I don't have the value to put in the emit..

Does anybody know how to do it?

menta man
  • 66
  • 9
  • The way that tools like Moles and Fakes do this is by intercepting the Il stream issued by the compiler using their own CLR profiler - http://stackoverflow.com/questions/3021797/how-moles-isolation-framework-is-implemented. There's a lot of stuff you'll need to take account of, and your major challenge is going to be mutating the code in a meaningful way (i.e. ensuring the il you write is valid C# - randomly mutating it without analytically based targeting won't really help you detect poor tests). I'd recommend trying to fix the error you're getting, rather than rolling your own. – Orphid Sep 23 '15 at 16:55
  • Thank you! I've tried fixing the error but I don't have any clue, I've downloaded the code from github and when running it I got "unknown custom metadata item kind 6" and I have no idea what it means, and I haven't managed to run it in debug mode. – menta man Sep 23 '15 at 17:01
  • Looking at the source code for VisualMutator, the exception you're talking about is thrown in this .cs: https://github.com/visualmutator/visualmutator/blob/4b114ab1121e814a1bf2e10336bd1ce5b2131b00/packages/CCI.1.0.13/src/Metadata/Sources/PdbReader/PdbFunction.cs, which seems to be reading the pdb file for an assembly. Without digging into it more, I can't give you too much advice on how to solve it. One useful bit of info I picked up reading some of the src was that it depends on Microsoft CCI - it may be worth checking out what you can do with that: https://ccimetadata.codeplex.com/. – Orphid Sep 23 '15 at 17:46
  • https://github.com/Fody/Fody/issues/141 I found this guy, which seems to have the same error with different project. He downgraded the cci package I think, and it fixed it for him, as far as I understood. I might give it a shot. – menta man Sep 23 '15 at 17:52
  • Ok I don't think he's talking about exactly the same thing, I'm already downgraded.. – menta man Sep 23 '15 at 18:06
  • btw in vs2013 I get this error: http://i.imgur.com/J6OiVSr.png it getter to further step in the testing. It happens when I start a testing session, it can't run the tests for a reason(and I can run them) – menta man Sep 23 '15 at 21:07
  • it works! I'm not sure why lol. maybe I still had a refrence for the MS testing framework instead of NUnit. thanks a lot! Just the only problem is that he doesn't kill the proccess and I can't save my files after starting a session. – menta man Sep 23 '15 at 21:14
  • Also it doesn't work with vs 2015 which really sucks.. cause I can't run it on projects that using c# 6 – menta man Sep 23 '15 at 21:52
  • Ok I made a version that works with 2015. I'll make a pull request – menta man Sep 23 '15 at 21:58

0 Answers0