Why can I not return a reference to a dictionary value?

Question

public class PropertyManager
{
    private Dictionary<ElementPropertyKey, string> _values = new Dictionary<ElementPropertyKey, string>();
    private string[] _values2 = new string[1];
    private List<string> _values3 = new List<string>();
    public PropertyManager()
    {
        _values[new ElementPropertyKey(5, 10, "Property1")] = "Value1";
        _values2[0] = "Value2";
        _values3.Add("Value3");
    }

    public ref string GetPropertyValue(ElementPropertyKey key)
    {
        return ref _values[key]; //Does not compile. Error: An expression cannot be used in this context because it may not be returned by reference.         
    }

    public ref string GetPropertyValue2(ElementPropertyKey key)
    {
        return ref _values2[0]; //Compiles
    }

    public ref string GetPropertyValue3(ElementPropertyKey key)
    {
        return ref _values3[0]; //Does not compile. Error: An expression cannot be used in this context because it may not be returned by reference.
    }
}

In the above example GetPropertyValue2 compiles, but GetPropertyValue and GetPropertyValue3 do not. What is wrong with returning a value from a dictionary or list as reference, while it does work for an array?

Reference semantics was introduced in 7.2 for value types. Why would you need `ref` in your case of returning a reference type? — Micha Wiedenmann, Jan 03 '18 at 11:28
@MichaWiedenmann, whilst you are correct, your comment is surely orthogonal to the question? Changing `string` to `int` in the OP's example code still yields the same errors. — David Arno, Jan 03 '18 at 11:38
@Lander you should be asking why you can use this with arrays when you can't use it with other containers. [Jon Skeet answered](https://stackoverflow.com/questions/48075085/why-can-i-ref-return-an-item-of-an-array-that-only-exists-inside-the-method/48075165#48075165) this earlier today. — Panagiotis Kanavos, Jan 03 '18 at 15:12
@Lander indexers on containers with value types return a copy of the value that gets allocated in the stack and will disappear once you exit the method. You can't return that safely with `return ref`. Array indexers though are provided by the CLR itself and return a reference to the value [as shown in this answer](https://stackoverflow.com/questions/6705583/indexers-in-list-vs-array) — Panagiotis Kanavos, Jan 03 '18 at 15:35
@PanagiotisKanavos, nowhere in Jon's answer to that other question can I find anything that explains why array elements can be used with ref returns and elements of other collections cannot. All he talks about is stack and heap allocations. A Dictionary is allocated on the heap... — David Arno, Jan 03 '18 at 16:01
@DavidArno that's explained inthe second question I linked to, [Indexers in List vs Arrays](https://stackoverflow.com/questions/6705583/indexers-in-list-vs-array). List/Dictionary indexers return stack-allocated copies thus unsafe. Array indexers are special beasts implemented by the CLR itself and return references to the data itself, not a copy of it. The important thing is that those references extend the array's lifetime as if they were fields — Panagiotis Kanavos, Jan 03 '18 at 16:09

Akos Nagy · Accepted Answer · 2018-01-04T09:54:00.860

I'd like to add my answer to the 'pot', maybe it makes things a bit more clear. So, why doesn't that work for lists and dictionaries? Well, if you have a piece of code like this:

static string Test()
{
   Dictionary<int, string> s = new Dictionary<int, string>();
   return s[0];
}

This (in debug mode) translates to this IL code:

IL_0000: nop
IL_0001: newobj instance void class [mscorlib]System.Collections.Generic.Dictionary`2<int32, string>::.ctor()
IL_0006: stloc.0
IL_0007: ldloc.0
IL_0008: ldc.i4.0
IL_0009: callvirt instance !1 class [mscorlib]System.Collections.Generic.Dictionary`2<int32, string>::get_Item(!0)
IL_000e: stloc.1
IL_000f: br.s IL_0011

IL_0011: ldloc.1
IL_0012: ret

This, in turn means that what you do with one line of code (return s[0]) is actually a three-step process: calling the method, storing the return value in a local variable and then returning the value that is stored in that local variable. And, as pointed out by the links the others have provided, returning a local variable by reference is not possible (unless the local variable is a ref local variable, but as pointed out by the others again, since Diciotionary<TKey,TValue> and List<T> does not have a by-reference return API, this is not possible either).

And now, why does it work for the array? If you look at how array-indexing is handled more closely (i.e. on IL-code level), you can see that there is no method call for array indexing. Instead, a special opcode is added to the code called ldelem (or some variant of that). A code like this:

 static string Test()
 {
    string[] s = new string[2];
    return s[0];
 }

translates to this in IL:

IL_0000: nop
IL_0001: ldc.i4.2
IL_0002: newarr [mscorlib]System.String
IL_0007: stloc.0
IL_0008: ldloc.0
IL_0009: ldc.i4.0
IL_000a: ldelem.ref
IL_000b: stloc.1
IL_000c: br.s IL_000e

IL_000e: ldloc.1
IL_000f: ret

Of course this looks like the same as it was for the dictionary, but I think the key difference is that the indexer here generates an IL-native call, not a property (i.e. method) call. And if you look at all the possible ldelem variants on MSDN here, you can see that there is one called ldelema which can load the address of the element directly to the heap. And indeed, if you write a piece of code like this:

static ref string Test()
{
   string[] s = new string[2];
   return ref s[0];
}

This translates to the following IL code, utilizing the direct-reference loading ldelema opcode:

IL_0000: nop
IL_0001: ldc.i4.2
IL_0002: newarr [mscorlib]System.String
IL_0007: stloc.0
IL_0008: ldloc.0
IL_0009: ldc.i4.0
IL_000a: ldelema [mscorlib]System.String
IL_000f: stloc.1
IL_0010: br.s IL_0012

IL_0012: ldloc.1
IL_0013: ret

So basically, array indexers are different and under the hood, arrays have support for loading an element by reference to the evaluation stack via native IL calls. Since the Dictionary<TKey,TValue> and other collections implement indexers as properties, which result in method calls, they can only do this if the method called explicitly specifies ref returns.

Thanks, this explained it to me. One remark: I suppose the second code block should be showing an array. Currently it's the same code as the first code block. — Coder14, Jan 04 '18 at 09:29
Thanks for the remark, your are absolutely right. Edited the code with the right snippet, — Akos Nagy, Jan 04 '18 at 09:55
I wonder if any future version of List/Dictionary will contain methods to return a value by reference, as that's now supported in the language. ref string = ref dictionary.GetValueByRef(1); Would be nice. — Coder14, Jan 04 '18 at 10:22

Why can I not return a reference to a dictionary value?

1 Answers1