Polymorphism with generics - strange behavior

Question

Pluggable framework

Imagine a simple pluggable system, which is pretty straightforward using inheritance polymorphism:

1. We have a graphics rendering system
2. There are different types of graphics shapes (monochrome, color, etc.) that need rendering
3. Rendering is done by a data-specific plugin, e.g. a ColorRenderer will render a ColorShape.
4. Every plugin implements IRenderer, so they can all be stored in an IRenderer[].
5. On startup, IRenderer[] is populated with a series of specific renderers
6. When data for a new shape is received, a plugin is chosen from the array based on the type of the shape.
7. The plugin is then invoked by calling its Render method, passing the shape as its base type.
8. The Render method is overridden in each descendant class; it casts the Shape back to its descendant type and then renders it.

Hopefully the above is clear-- I think it is a pretty common sort of setup. Very easy with inheritance polymorphism and run-time casting.

Doing it without casting

Now the tricky part. In response to this question, I wanted to come up with a way to do this all without any casting whatsoever. This is tricky because of that IRenderer[] array-- to get a plugin out of the array, you would normally need to cast it to a specific type in order to use its type-specific methods, and we can't do that. Now, we could get around that by interacting with a plugin only with its base class members, but part of the requirements was that the renderer must run a type-specific method that has a type-specific data packet as an argument, and the base class would not be able to do that because there is no way to pass it a type-specific data packet without a casting it to the base and then back to the ancestor. Tricky.

At first I thought it was impossible, but after a few tries I found I could make it happen by juking the c# generic system. I create an interface that is contravariant with respect to both plugin and shape type and then used that. Resolution of the renderer is decided by the type-specific Shape. Xyzzy, the contravariant interface makes the cast unnecessary.

Here is the shortest version of the code I could come up with as an example. This compiles and runs and behaves correctly:

public enum ColorDepthEnum { Color = 1, Monochrome = 2 }

public interface IRenderBinding<in TRenderer, in TData> where TRenderer : Renderer 
                                                  where TData: Shape  
{ 
    void Render(TData data);
}
abstract public class Shape
{
    abstract public ColorDepthEnum ColorDepth { get; }
    abstract public void Apply(DisplayController controller);
}

public class ColorShape : Shape
{
    public string TypeSpecificString = "[ColorShape]";  //Non-virtual, just to prove a point
    override public ColorDepthEnum ColorDepth { get { return ColorDepthEnum.Color; } }

    public override void Apply(DisplayController controller)
    {
        IRenderBinding<ColorRenderer, ColorShape> renderer = controller.ResolveRenderer<ColorRenderer, ColorShape>(this.ColorDepth);
        renderer.Render(this);
    }
}
public class MonochromeShape : Shape
{
    public string TypeSpecificString = "[MonochromeShape]";  //Non-virtual, just to prove a point
    override public ColorDepthEnum ColorDepth { get { return ColorDepthEnum.Monochrome; } }

    public override void Apply(DisplayController controller)
    {
        IRenderBinding<MonochromeRenderer, MonochromeShape> component = controller.ResolveRenderer<MonochromeRenderer, MonochromeShape>(this.ColorDepth);
        component.Render(this);
    }
}


abstract public class Renderer : IRenderBinding<Renderer, Shape>
{
    public void Render(Shape data) 
    {
        Console.WriteLine("Renderer::Render(Shape) called.");
    }
}


public class ColorRenderer : Renderer, IRenderBinding<ColorRenderer, ColorShape>
{

    public void Render(ColorShape data) 
    {
        Console.WriteLine("ColorRenderer is now rendering a " + data.TypeSpecificString);
    }
}

public class MonochromeRenderer : Renderer, IRenderBinding<MonochromeRenderer, MonochromeShape>
{
    public void Render(MonochromeShape data)
    {
        Console.WriteLine("MonochromeRenderer is now rendering a " + data.TypeSpecificString);
    }
}


public class DisplayController
{
    private Renderer[] _renderers = new Renderer[10];

    public DisplayController()
    {
        _renderers[(int)ColorDepthEnum.Color] = new ColorRenderer();
        _renderers[(int)ColorDepthEnum.Monochrome] = new MonochromeRenderer();
        //Add more renderer plugins here as needed
    }

    public IRenderBinding<T1,T2> ResolveRenderer<T1,T2>(ColorDepthEnum colorDepth) where T1 : Renderer where T2: Shape
    {
        IRenderBinding<T1, T2> result = _renderers[(int)colorDepth];  
        return result;
    }
    public void OnDataReceived<T>(T data) where T : Shape
    {
        data.Apply(this);
    }

}

static public class Tests
{
    static public void Test1()
    {
       var _displayController = new DisplayController();

        var data1 = new ColorShape();
        _displayController.OnDataReceived<ColorShape>(data1);

        var data2 = new MonochromeShape();
        _displayController.OnDataReceived<MonochromeShape>(data2);
    }
}

If you run Tests.Test1() the output will be:

ColorRenderer is now rendering a [ColorShape]
MonochromeRenderer is now rendering a [MonochromeShape]

Beautiful, it works, right? Then I got to wondering... what if ResolveRenderer returned the wrong type?

Type safe?

According to this MSDN article,

Contravariance, on the other hand, seems counterintuitive....This seems backward, but it is type-safe code that compiles and runs. The code is type-safe because T specifies a parameter type.

I am thinking, there is no way this is actually type safe.

Introducing a bug that returns the wrong type

So I introduced a bug into the controller so that is mistakenly stores a ColorRenderer where the MonochromeRenderer belongs, like this:

public DisplayController()
{
    _renderers[(int)ColorDepthEnum.Color] = new ColorRenderer();
    _renderers[(int)ColorDepthEnum.Monochrome] = new ColorRenderer(); //Oops!!!
}

I thought for sure I'd get some sort of type mismatch exception. But no, the program completes, with this mysterious output:

ColorRenderer is now rendering a [ColorShape]
Renderer::Render(Shape) called.

What the...?

My questions:

First,

Why did MonochromeShape::Apply call Renderer::Render(Shape)? It is attempting to call Render(MonochromeShape), which obviously has a different method signature.

The code within the MonochromeShape::Apply method only has a reference to an interface, specifically IRelated<MonochromeRenderer,MonochromeShape>, which only exposes Render(MonochromeShape).

Although Render(Shape) looks similar, it is a different method with a different entry point, and isn't even in the interface being used.

Second,

Since none of the Render methods are virtual (each descendant type introduces a new, non-virtual, non-overridden method with a different, type-specific argument), I would have thought that the entry point was bound at compile time. Are method prototypes within a method group actually chosen at run-time? How could this possibly work without a VMT entry for dispatch? Does it use some sort of reflection?

Third,

Is c# contravariance definitely not type safe? Instead of an invalid cast exception (which at least tells me there is a problem), I get an unexpected behavior. Is there any way to detect problems like this at compile time, or at least to get them to throw an exception instead of doing something unexpected?

Answer 1

OK, first of all, do not write generic types like this. As you have discovered, it rapidly becomes a huge mess. Never do this:

class Animal {}
class Turtle : Animal {}
class BunchOfAnimals : IEnumerable<Animal> {}
class BunchOfTurtles : BunchOfAnimals, IEnumerable<Turtle> {}

OH THE PAIN. Now we have two paths by which to get an IEnumerable<Animal> from a BunchOfTurtles: Either ask the base class for its implementation, or ask the derived class for its implementation of the IEnumerable<Turtle> and then covariantly convert it to IEnumerable<Animal>. The consequences are: you can ask a bunch of turtles for a sequence of animals, and giraffes can come out. That's not a contradiction; all the capabilities of the base class are present in the derived class, and that includes generating a sequence of giraffes when asked.

Let me re-emphasize this point so that it is very clear. This pattern can create in some cases implementation-defined situations where it becomes impossible to determine statically what method will actually be called. In some odd corner cases, you can actually have the order in which the methods appear in the source code be the deciding factor at runtime. Just don't go there.

For more on this fascinating topic I encourage you to read all the comments to my 2007 blog post on the subject: https://blogs.msdn.microsoft.com/ericlippert/2007/11/09/covariance-and-contravariance-in-c-part-ten-dealing-with-ambiguity/

Now, in your specific case everything is nicely well defined, it's just not defined as you think it ought to be.

To start with: why is this typesafe?

IRenderBinding<MonochromeRenderer, MonochromeShape> component = new ColorRenderer();

Because you said it should be. Work it out from the point of view of the compiler.

• A ColorRenderer is a Renderer
• A Renderer is a IRenderBinding<Renderer, Shape>
• IRenderBinding is contravariant in both its parameters, so it may always be made to have a more specific type argument.
• Therefore a Renderer is an IRenderBinding<MonochromeRenderer, MonochromeShape>
• Therefore the conversion is valid.

Done.

So why is Renderer::Render(Shape) called here?

    component.Render(this);

You ask:

Since none of the Render methods are virtual (each descendant type introduces a new, non-virtual, non-overridden method with a different, type-specific argument), I would have thought that the entry point was bound at compile time. Are method prototypes within a method group actually chosen at run-time? How could this possibly work without a VMT entry for dispatch? Does it use some sort of reflection?

Let's go through it.

component is of compile-time type IRenderBinding<MonochromeRenderer, MonochromeShape>.

this is of compile-time type MonochromeShape.

So we are calling whatever method implements IRenderBinding<MonochromeRenderer, MonochromeShape>.Render(MonochromeShape) on a ColorRenderer.

The runtime must figure out which interface is actually meant. ColorRenderer implements IRenderBinding<ColorRenderer, ColorShape> directly and IRenderBinding<Renderer, Shape> via its base class. The former is not compatible with IRenderBinding<MonochromeRenderer, MonochromeShape>, but the latter is.

So the runtime deduces that you meant the latter, and executes the call as though it were IRenderBinding<Renderer, Shape>.Render(Shape).

So which method does that call? Your class implements IRenderBinding<Renderer, Shape>.Render(Shape) on the base class so that's the one that's called.

Remember, interfaces define "slots", one per method. When the object is created, each interface slot is filled with a method. The slot for IRenderBinding<Renderer, Shape>.Render(Shape) is filled with the base class version, and the slot for IRenderBinding<ColorRenderer, ColorShape>.Render(ColorShape) is filled with the derived class version. You chose the slot from the former, so you get the contents of that slot.

Is c# contravariance definitely not type safe?

I promise you it is type safe. As you should have noticed: every conversion you made without a cast was legal, and every method you called was called with something of a type that it expected. You never invoked a method of ColorShape with a this referring to a MonochromeShape, for instance.

Instead of an invalid cast exception (which at least tells me there is a problem), I get an unexpected behavior.

No, you get entirely expected behaviour. You just have created a type lattice that is extraordinarily confusing, and you don't have a sufficient level of understanding of the type system to understand the code you wrote. Don't do that.

Is there any way to detect problems like this at compile time, or at least to get them to throw an exception instead of doing something unexpected?

Don't write code like that in the first place. Never implement two versions of the same interface such that they may unify via covariant or contravariant conversions. It is nothing but pain and confusion. And similarly, never implement an interface with methods that unify under generic substitution. (For example, interface IFoo<T> { void M(int); void M(T); } class Foo : IFoo<int> { uh oh } )

I considered adding a warning to that effect, but it was difficult to see how to turn off the warning in the rare cases where it is desirable. Warnings that can only be turned off with pragmas are poor warnings.

Answer 2

First. MonochromeShape::Apply call Renderer::Render(Shape) because of the following:

IRenderBinding<ColorRenderer, ColorShape> x1 = new ColorRenderer();
IRenderBinding<Renderer, Shape> x2 = new ColorRenderer();
// fails - cannot convert IRenderBinding<ColorRenderer, ColorShape> to IRenderBinding<MonochromeRenderer, MonochromeShape>
IRenderBinding<MonochromeRenderer, MonochromeShape> c1 = x1;
// works, because you can convert IRenderBinding<Renderer, Shape> toIRenderBinding<MonochromeRenderer, MonochromeShape>
IRenderBinding<MonochromeRenderer, MonochromeShape> c2 = x2;

So in short: ColorRenderer inherits from Renderer and that in turn implements IRenderBinding<Renderer, Shape>. This interface is what allows ColorRendered to be implicitly converted to IRenderBinding<MonochromeRenderer, MonochromeShape>. This interface is implemented by class Renderer and so it's not suprising that Renderer.Render is called when you call MonochromeShape::Apply. The fact you pass instance of MonochromeShape and not Shape is not a problem exactly because TData is contravariant.

About your second question. Dispatch by interface is virtual just by definition. In fact, if method implements some method from interface - it's marked as virtual in IL. Consider this:

class Test : ITest {
    public void DoStuff() {

    }
}

public class Test2 {
    public void DoStuff() {

    }
}

interface ITest {
    void DoStuff();
}

Method Test.DoStuff has following signature in IL (note virtual:

.method public final hidebysig virtual newslot instance void 
    DoStuff() cil managed 

Method Test2.DoStuff is just:

.method public hidebysig instance void 
    DoStuff() cil managed

As for third question I think it's clear from above that it behaves as expected and is type-safe exactly because no invalid cast exceptions are possible.