C# params apparent compiler bug (C# 5.0)

Question

This is a followup on a thread I thought was resolved yesterday. Yesterday I was having problems with my code in the following case:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace ConsoleApplication3
{
    class Program
    {
        class Bar
        {
            int v;

            public Bar(int v) { this.v = v; }
            public override string ToString() { return v.ToString(); }
        }

        static void Main(string[] args)
        {
            Foo(1, 2, 3);
            Foo(new int[] { 1, 2, 3 });
            Foo(new Bar(1), new Bar(2), new Bar(3));
            Foo(new Bar[] { new Bar(1), new Bar(2), new Bar(3) });
            System.Threading.Thread.Sleep(20000);
        }

        static void Foo(params object[] objs)
        {
            Console.WriteLine("New call to Foo: ");
            foreach(object o in objs)
                Console.WriteLine("Type = " + o.GetType() + ", value = "+o.ToString());
        }
    }
}

If you run this you can see a problem with the last call to Foo. The fact that the argument is a vector is "lost".

So.... anyone know how to report a C# compiler bug? Or would this be considered a reflection bug?

(What a relief: I was bummed to think I had wasted time here with a bug of my own. In fact it is a C# bug after all, and I'm vindicated! And how often do we get to see actual C# compiler bugs these days? Not common...)

I linked what looks like the question from yesterday you are referring to. If it's the wrong one, please include a link to the correct one. — M.Babcock, Mar 14 '12 at 20:29
There's a bug in your code, one right parenthesis too much at `new Foobar(3))`, isn't it? — Tim Schmelter, Mar 14 '12 at 20:34
Done. If you run this example you'll see that the last call to Foo malfunctions in the way I described. BTW: Why does my code misformat? Can someone fix it? — Ken Birman, Mar 14 '12 at 20:41
Ken, to create a code block in your post, you need to indent each line of code with at least four spaces. You can use the `{}` button to do that to a large block of text. — phoog, Mar 14 '12 at 20:45
It's pretty unclear what you consider the bug, but it doesn't work any differently in earlier compiler versions. Try it. If you want just a single argument to be passed then you'll have to hit the compiler over the head with Foo(new object[] { new FooBar[] {...} }); — Hans Passant, Mar 14 '12 at 20:53
@HansPassant It's easier to simply cast to `object`, as in `Foo((object)new Bar[] { new Bar(1), new Bar(2), new Bar(3) });`. As far as I know, it is always possible to disambiguate between the "normal form" and the "expanded" form of a `params` method with just a cast, so it's never necessary to explicitly create the outer-most array. — Jeppe Stig Nielsen, Apr 02 '13 at 20:33

score 18 · Answer 1 · answered Mar 14 '12 at 20:54

The C# 4.0 specification is quite explicit in how this all plays out. 7.5.3.1 says that if a function with params can be applied either in normal form (ignoring the params keyword) or expanded form (using the params keyword), then normal form wins.

Assuming Foo was declared as Foo(params object[] args), then the call Foo(new Foobar[] {new Foobar(1), new Foobar(2), new Foobar(3)) }); is applicable in normal form, since Foobar[] is implicitly convertible to object[] (6.1.6 clause 5). Therefore, the normal form is used and the expanded form is ignored.

(I'm assuming that C# 5.0 did not change this part of the language.)

score 7 · Accepted Answer · edited Feb 27 '14 at 08:02

7

I would expect these two calls to function identically- a params argument is an array in the called method. Jon Skeet's example in the previous question works because an array of int's is not covariant to an array of objects (and so is treated as new Object[] { new Int[] {1,2,3} }), but in this example an array of FooBars is covariant to an array of objects, and so your parameter is expanded into the objs argument.

Wikipedia of all things covers this exact case: Covariance and contravariance (computer science)

Sorry but I am sure that this is not a compiler bug.

EDIT:

You can achieve what you want thus:

Foo(new Object[] { new Bar[] { new Bar(1), new Bar(2), new Bar(3) } });

NEW EDIT (other author):

Or simply use:

Foo((Object)new Bar[] { new Bar(1), new Bar(2), new Bar(3) });

edited Feb 27 '14 at 08:02

AminM

1,658
4
32
48

answered Mar 14 '12 at 20:42

Chris Shain

50,833
6
93
125

I love PL people. But in this particular case they clearly have talked themselves into a mistake! – Ken Birman Mar 14 '12 at 20:43
Sorry- I don't understand that last comment – Chris Shain Mar 14 '12 at 20:44
Clearly there should be a way to use params to figure out exactly what the argument to a method was. By adopting this definition, the C# 5.0 guys have created a case that cannot be correctly handled using params. So while you are clearly right (just read the Wikipedia page) the definition being used is clearly wrong! Anyhow, is there a way to do what I actually am trying to do? – Ken Birman Mar 14 '12 at 20:47
1

@KenBirman - I think this really comes down to why do you need to know exactly what it was called with? I can not think of many practical situations where this is the case. For the cases where it is needed, you can do that via the Dynamic namespace as Jon Skeet pointed out – John Mar 14 '12 at 20:49
1

In Isis2, people build parallel applications. Suppose someone queries a set of servers and each returns, say, URLs that match some pattern (each searching a separate corpus). Some find 2 matches, some 4. I'm unable to extract the type signature hence unable to return sensible lists of responses for the caller. In effect, I can't distinguish g.Reply("url1", "url2") from g.Reply(vector-of-urls); Of course I support the general case, not just strings. – Ken Birman Mar 14 '12 at 20:58
I'm looking more closely at what Jon Skeet suggested. If that solves my issue, I'm good. – Ken Birman Mar 14 '12 at 21:00
In that case you should eliminate the use of `params` and force the caller to be more specific. This usage that you are proposing is incredibly confusing, especially considering the difference between value and reference types. Either that, or take Hadoop's approach and only allow a single return value, which may be of a complex type if needed. Maybe have a look at the Tuple class: http://msdn.microsoft.com/en-us/library/system.tuple.aspx?ppud=4 – Chris Shain Mar 14 '12 at 21:02
OK, I see how Jon's thing works. Forces me to make Isis2 groups Dynamic and forces my users to employ the dynamic keyword, hence groddy and undesireable. Even more undesireable is the suggestion above, although yes, it works. C# users shouldn't need to understand covariance on object arrays to use the language! I still say that this is a bug (a bug in the language definition) – Ken Birman Mar 14 '12 at 21:04
Oh, and Chris, since I marshall into messages, all my stuff will be by value -- I take what they provide and copy it into a message and send it to the guy who did g.Query. It looks very elegant and actually, Isis2 is WAY easier to use than most other such packages. This is why the original Isis was so popular way back in the old days, too. My users are the kind of people who program mouse click handlers (and that's the API model I adopted). – Ken Birman Mar 14 '12 at 21:06
C# users SHOULD need to understand reference array covariance, and the type safety hiccups that come with it. If you don't understand it, then use ArrayList or List instead. Both are non co-variant (ArrayList for obvious reasons). That is why it is considered a best practice not to involve [] arrays as part of an API. – Michael Graczyk Jul 07 '12 at 00:37

phoog · Answer 3 · 2012-03-14T21:06:21.520

3

See section 7.5.3.1 of the C# spec:

7.5.3.1 Applicable function member

A function member is said to be an applicable function member with respect to an argument list A when all of the following are true:

Each argument in A corresponds to a parameter in the function member declaration as described in §7.5.1.1, and any parameter to which no argument corresponds is an optional parameter.

For each argument in A, the parameter passing mode of the argument (i.e., value, ref, or out) is identical to the parameter passing mode of the corresponding parameter, and

for a value parameter or a parameter array, an implicit conversion (§6.1) exists from the argument to the type of the corresponding parameter, or

[... some irrelevant material concerning ref and out parameters ...]

For a function member that includes a parameter array, if the function member is applicable by the above rules, it is said to be applicable in its normal form. If a function member that includes a parameter array is not applicable in its normal form, the function member may instead be applicable in its expanded form[.]

Because the array you passed can be implicitly converted to object[], and because overload resolution prefers "normal" form over "expanded" form, the behavior you observe conforms to the specification, and there is no bug.

In addition to the workaround described by Chris Shain, you could also change Bar from a class to a struct; that makes the array no longer implicitly convertible to object[], so you'll get the behavior you desire.

edited Mar 14 '12 at 21:06

answered Mar 14 '12 at 20:54

phoog

42,068
6
79
117

I'm accepting that the compiler implements the spec. But the spec clearly is buggy, as this example is illustrating in greater and greater clarity. How can it be that g.Reply(vector-of-Foos) is indistinguishable from g.Reply(Foo,Foo'...) and yet g.Reply(3, vector-of-foos) works as one would expect? Yes, I see why this works, any of us on this thread see that. But why in a profound philosophical sense is this the correct decision on how to infer the type? They had two paths in the woods... and took the wrong one. Then made it part of the spec. Still a mistake! – Ken Birman Mar 14 '12 at 21:13
1

@KenBirman it seems to me that the solution to your problem is to abandon the use of `params` and just take a single `object[]` argument; require your users to package the arguments into an array no matter what. Then there's no ambiguity. Also, I'd question the use of "buggy" here; if the language designers made a poor choice, then by all means call it a "poor choice". Certainly, the system as it works now has several useful properties. – phoog Mar 14 '12 at 21:19
5

@KenBirman: Design is always the process of compromising between numerous incompatible goals. What appears to you to be a "mistake" is certainly an unfortunate consequence of that design process, but you are not taking into account all the considerations that the designers had to take into account. Fundamentally the problem arises from the fact that unsafe array covariance only works on reference types. This is unfortunate, yes, but it is not a bug or a mistake; this was deliberately designed into the language by sensible people who had to make hard compromises. – Eric Lippert Mar 14 '12 at 21:27
@EricLippert - well, I'd say that in hindsight array covariance is a mistake in the CLR. From the C# perspective it's more justifiable since the underlying platform supports it (and matching the platform has some benefit). – kvb Mar 14 '12 at 21:40
Eric, see below. By now I believe the spec is correct and the type match too, but that this is a boring compiler bug. I think there just is no way to parse the "params" part of the type signature in which this covariance issue even arises. The compiler is messing up by ignoring params; in fact, if you think about what it did do, it clearly was buggy behavior! (But see below; running out room for comments on this particular proposed "answer") – Ken Birman Mar 14 '12 at 21:46
@kvb: I agree with you. However, the v1.0 CLR designers also lived in a world with conflicting goals and had to choose a reasonable compromise design. I am right there wishing that this decision had been made differently; my opinion is that the benefit of the feature does not justify the pain it also causes. But just because I selfishly wish that we'd chosen differently over a decade ago does not make that decision a "mistake" or a "bug". The decision was undertaken with a full understanding of the consequences. – Eric Lippert Mar 14 '12 at 21:47
@kvb that's true, but as I understand it, the CLR supports it because it was a desired feature for C#; it was a desired feature for C# because it was a feature of Java. If that's true, the underlying mistake was indeed one of C# language design. – phoog Mar 14 '12 at 21:51
@KenBirman: Your analysis is incorrect; this is not a C# compiler bug. – Eric Lippert Mar 14 '12 at 21:52
3

@phoog: No, you are getting it backwards. The CLR supports it because the CLR wanted to be able to be the runtime for any possible Java-like language, so it supports a superset of the features of Java. Given that the CLR's common type system supports unsafe array covariance it is then natural for C# to support it as well; it would be a bit weird otherwise. (However, C# does not support all the covariance features of the common type system; for example, a cast from int[] to uint[] is illegal in C#, even though the CLR allows that conversion.) – Eric Lippert Mar 14 '12 at 21:55
@EricLippert thanks for clearing that up. – phoog Mar 14 '12 at 21:58
@phoog: You're welcome. An interesting consequence of the mismatch between the C# type system and the CLR type system is that the `is` operator behaves quite oddly in some of these corner cases. See http://stackoverflow.com/questions/593730/why-does-int-is-uint-true-in-c-sharp for example. – Eric Lippert Mar 14 '12 at 22:05
@EricLippert - I certainly agree that it's unfair to look back at decisions made in the past and criticize them as if they were being made today, and I understand that in the .NET 1.0 context array covariance may have made sense. Perhaps I'm just arguing semantics, but I'd still classify it as a mistake (in hindsight), and I'd disagree that it was made "with a full understanding of the consequences" (or else why do we feel differently today?). – kvb Mar 15 '12 at 01:49

Kamyar Nazeri · Answer 4 · 2012-03-14T20:58:20.443

2

I believe you are wrong Ken That's because int[] is not of object[] type, so the compiler assumes that the int[] is just one of arguments passed to the method.

here's how:

new Foobar[] { } is object[]; // true
new int[] { } is object[]; // false

update: you can make the method generic to let the compiler know struct/object type passes as params:

void Foo<T>(params T[] objs)
{
    foreach (T o in objs)
        Console.WriteLine(o.GetType());
}

edited Mar 14 '12 at 20:58

answered Mar 14 '12 at 20:46

Kamyar Nazeri

25,786
15
50
87

Params is doing the type inference the other way around, Kamar! If your explanation held, then the 3rd, not the 4th, call would be the one that prints an undesired output – Ken Birman Mar 14 '12 at 20:50
this is how: in C# an instance of a struct is an object, however array of struct is not of type object[] – Kamyar Nazeri Mar 14 '12 at 20:56
Your best bet is to make the method generic, in that case compiler would decide on parameter type (struct/object) on the method call! check out the update – Kamyar Nazeri Mar 14 '12 at 20:57
Assumes (incorrectly) that the Reply is all of the same type. I often see, for example, g.Reply(quality, URL-list, ...). But this would work correctly. In fact the issue ONLY arises with a single argument that is a vector of user-defined objects and to me this highlights the absurdity of claiming that C# is correct to use this definition. – Ken Birman Mar 14 '12 at 21:09

score -6 · Answer 5 · answered Mar 14 '12 at 21:37

-6

So I'm going to suggest that the best answer is that I'm right and that this really is a compiler bug. While I see the point perfectly clearly, your explanation ignores the "params" keyword. In fact to use covariance this way, one MUST ignore the params keyword, as if it was unimportant. I postulate that there simply is no plausible explanation for compiling the code this way with Params present as a keyword on the type signature of Foo: to invoke your explanation you need to convince me that myClass[] should be type-matched to object[], but we shouldn't even be asking that question given the params construct. In fact, the more you think about this, the more clear that it is actually a genuine C# 5.0 compiler bug: the compiler is neglecting to apply the params keyword. The language spec doesn't actually need any change at all. I should get some kind of wierd badge, imho!

answered Mar 14 '12 at 21:37

Ken Birman

1,088
8
25

Your analysis is incorrect; this is not a compiler bug. All of the other analyses are correct; Raymond Chen's answer is in my opinion the most clear of all of them. – Eric Lippert Mar 14 '12 at 21:48
I'm not sure I follow your argument. On the one hand, you find it absurd that the `params` keyword is ignored, but on the other hand agreeing that spec is correct. But those points are at odds, since the spec makes it (relatively) clear that the interaction between `params` and an array of reference type is *exactly* what the compiler is producing. So how can that be a compiler bug? As you mentioned earlier, this behavior may be undesirable (though I feel that in most circumstances it shouldn't present a real problem,) but the compiler is implemented according to spec. – dlev Mar 14 '12 at 21:53
Can you tell us which part of the spec you believe is violated by the compiler's behavior? – phoog Mar 14 '12 at 21:56
Phoog, to give a good answer to your question will take me a little time. But my point, again, is that in order to apply Raymond's or the other suggested interpretations, we need to pretend that the invocation to Foo was identical to an invocation of a method not using the params keyword. But with params present, we take the next type argument to Foo (object[]) and match the arguments against type "object". That is, not against object[]. Incorrectly, C# matched against object[] and applied covariance. – Ken Birman Mar 14 '12 at 22:05
2

@KenBirman: It *is* identical to an invocation of a method not using the params keyword. As Raymond correctly said, if a method is applicable in both its *normal* (ignore params) and its *expanded* (use params) form then *the normal form wins*. (Also, be careful. I suspect that "type argument" does not mean what you think it means.) – Eric Lippert Mar 14 '12 at 22:10
2

@KenBirman: Perhaps a justification will help. `int M(bool b, params string[] p) { return b ? N(p) : O(p); } int N(params object[] q) {...} int O(params object[] r) {...}`. The calls to N and O are applicable *in their normal form*. Obviously it would be a bug here to pass `new object[] { p }` as the argument to N or O. If we do not allow you to call N or O *in normal form* then *it becomes impossible to write the method M*. – Eric Lippert Mar 14 '12 at 22:17
Hold on... thinking more about your example – Ken Birman Mar 14 '12 at 22:20
Eric, this example helps me see why C# does what it does. Let me think more about whether I agree that it is impossible to resolve things in a way that would make this example work AND give me a way to extract the actual type signature from the argument list without this absurdity -- one that in my view is as much of an issue as your impossibility example. Perhaps more so: N and O don't actually "need" the params keyword here. Essentially, you've provided a shortcut (no need to do a second version of N and O, if direct params calls are needed) that breaks my code "absolutely" (no way out) – Ken Birman Mar 14 '12 at 22:26
Those of you who voted this down just aren't thinking hard about the issue. Eric does get it (and maybe only Eric) but he and I just disagree on how C# should handle it. – Ken Birman Mar 15 '12 at 12:51
1

Ken, in @EricLippert's example, N and O might be part of the class's public surface, and might need `params` for that reason, or might be library methods outside the programmer's control. The way out would be to consider the argument list `(params object[] x)` to be distinct from `(object[] x)` for the purpose of overload resolution. I suspect that would introduce more cost than benefit. For one thing, the params version of every such method would look the same: `T M(params object[] a) { return M(a); } T M(object[] a) { // ... real logic here ...` – phoog Mar 15 '12 at 14:35
Ken, thinking further: with such a pair of overloads, how do you decide which one to call when the call site looks like `T result = M(new object[0]);`? – phoog Mar 15 '12 at 14:36
2

@KenBirman, you are free to disagree how C# should behave. But that does not make the current behavior a bug. – svick Apr 03 '12 at 11:31

C# params apparent compiler bug (C# 5.0)

5 Answers5

Linked