Anonymous delegate not using new local for every iteration when data on local stackalloc

Question

When using anonymous delegates in C# the CLR will generate a copy of the local (e.g. variables in the current scope) on the heap for used variables. Such a local will be put onto the heap for every declared variable of the current scope.

You can see this behavior on this example:

class Program
{
    static void Main(string[] args)
    {
        for (int i = 0; i < 5; i++)
            ThreadPool.QueueUserWorkItem(delegate { execute(i); });

        Thread.Sleep(1000);

        Console.WriteLine();

        for (int i = 0; i < 5; i++)
        {
            int j = i;

            ThreadPool.QueueUserWorkItem(delegate { execute(j); });
        }

        Thread.Sleep(1000);
    }

    static void execute(int number)
    {
        Console.WriteLine(" * NUM=" + number.ToString());
    }
}

The output of this program is (the order may vary on the last 5 entries while on the first one it's also possible to get lower numbers than 5.):

 * NUM=5
 * NUM=5
 * NUM=5
 * NUM=5
 * NUM=5

 * NUM=0
 * NUM=1
 * NUM=2
 * NUM=3
 * NUM=4

C# should always generate a new copy of the local when called in a method. This is working as intended in this example:

class Program
{
    static void Main(string[] args)
    {
        for (int i = 0; i < 5; i++)
            call(i);

        Thread.Sleep(1000);
    }

    static void call(int number)
    {
        ThreadPool.QueueUserWorkItem(delegate { execute(number); });
    }

    static void execute(int number)
    {
        Console.WriteLine(" * NUM=" + number.ToString());
    }
}

Output:

 * NUM=0
 * NUM=1
 * NUM=2
 * NUM=3
 * NUM=4

This is the case in question: However, it's not working when assigning the variable to a stackalloc reserved area:

class Program
{
    static void Main(string[] args)
    {
        for (int i = 0; i < 5; i++)
            call(i);

        Thread.Sleep(1000);
    }

    static unsafe void call(int number)
    {
        int* ints = stackalloc int[64];

        ints[32] = number;

        ThreadPool.QueueUserWorkItem(delegate { execute(ints[32]); });
    }

    static void execute(int number)
    {
        Console.WriteLine(" * NUM=" + number.ToString());
    }
}

Output:

 * NUM=4
 * NUM=4
 * NUM=4
 * NUM=4
 * NUM=4

When using a regular local variable - just replace the call method from the example above:

static void call(int number)
{
    int j = number;

    ThreadPool.QueueUserWorkItem(delegate { execute(j); });
}

Output:

 * NUM=0
 * NUM=1
 * NUM=2
 * NUM=3
 * NUM=4

This situation makes me not trust anonymous delegates in C# - because I don't understand when exactly C# won't fuck up my calls to anonymous delegates.

My Questions: Why does C# not keep track of the stackalloc space regarding anonymous delegates? I'm aware of C# not keeping track. I want to know why it's not keeping track, if it does with a regular variable.

I used .NET Core 2.1 with C# 7.3 including /unsafe switch for those examples.

Answer 1

The problem is that you're capturing a pointer. That pointer refers to memory that is allocated on the stack by call - and the pointer keeps a reference to it even after the method has returned, which is fundamentally bad news. You're into undefined territory at that point - there's no guarantee what will be in that memory later.

Each stackalloc does occur separately - the five pointers you've got are all independent, but they happen to refer to the same piece of memory because each is the result of a separate stackalloc execution when the stack pointer is at the same value to start with. You can still use that memory because it's still valid memory within the process, but it's not safe to do so in terms of knowing what's going to be there.

The ints variable is "correctly" being copied into a class generated by the compiler, but the value of the variable refers to memory which was on the stack at the time of the call to the call method. When I ran the code, I got output of "whatever the argument to Thread.Sleep was. The C# compiler is capturing the variables, which isn't the same as "capturing the entire content of the stack".

You don't need to avoid delegates entirely - you just need to avoid mixing delegates with unsafe code and stack allocation.

You can see this problem without using any anonymous functions at all:

using System;
using System.Threading;

class Program
{
    static void Main(string[] args)
    {
        for (int i = 0; i < 5; i++)
        {
            Call(i);
        }

        Thread.Sleep(999);
    }

    static unsafe void Call(int number)
    {
        Helper helper = new Helper();
        int* tmp = stackalloc int[64];
        helper.ints = tmp;
        helper.ints[32] = number;        
        ThreadPool.QueueUserWorkItem(helper.Method);
    }

    static void Execute(int number)
    {
        Console.WriteLine(" * NUM=" + number.ToString());
    }

    unsafe class Helper
    {
        public int* ints;

        public void Method(object state)            
        {
            Execute(ints[32]);
        }
    }    
}

You can see it easily without using any delegates, but doing the same thing of "stack allocate some memory, and use a pointer to it after that stack has gone away":

using System;

class Program
{
    unsafe static void Main(string[] args)
    {
        int*[] pointers = new int*[5];
        for (int i = 0; i < 5; i++)
        {
            pointers[i] = Call(i);
        }
        for (int i = 0; i < 5; i++)
        {
            Console.WriteLine(pointers[i][32]);
        }
    }

    unsafe static int* Call(int number)
    {
        int* ints = stackalloc int[64];
        ints[32] = number;
        return ints;
    }
}