Parameter value is incremented in delegate

Question

I'm still learning how to use multi-threading and I have a weird behavior. When executing the following code without break points, the value passed to the thread function is incremented (If i=0, x will be 1) I don't know why.

static void Main(string[] args)
{
    for (int i = 0; i < 5; i++)
    {
        Thread t = new Thread(() => WriteNumbers(i));
        t.Start();
        // Console.WriteLine(i);
    }
    Console.ReadLine();
}
static void WriteNumbers(int x)
{
    string c = "";
    switch (x)
    {
        case 0:
            c = " A ";
            break;
        case 1:
            c = " B ";
            break;
        case 2:
            c = " C ";
            break;
        case 3:
            c = " D ";
            break;
        case 4:
            c = " E ";
            break;
    }
    for (int j = 0; j < 5; j++)
    {
        Console.Write(c);
    }
}

However, if I un-comment WriteLine(i). The delegate function will receive the parameter correctly.

EDIT:

The answer in Anonymous c# delegate within a loop & https://blogs.msdn.microsoft.com/ericlippert/2009/11/12/closing-over-the-loop-variable-considered-harmful/ doesn't explain 2 things: 1- Why NOT all the values of X are 5? I get some 4s and some 5s. 2- Why adding Console.WriteLine(i) fixes the problem.