Let's see the assembly:
-O0: (No optimizations,gcc, just a bit stripped):
main:
pushq %rbp #Push address to stack
movq %rsp, %rbp
movl $0, %eax
call main
movl $0, %eax
popq %rbp
ret
You just call a function recursively, always pushing the return address to the stack. But after some implementation/platform defined number of calls there is another memory. You just overwrite it or you access invalid memory. (Segmentation fault)
You can see it like this:
High=======Lower addresses
[stack].....[someOtherMemory]
...recursive call...
[stack ]...[someOtherMemory]
...recursive call...
[stack ].[someOtherMemory]
...recursive call...
[stack ]someOtherMemory]
Kaboom: You accessed the other memory, that was maybe not writable/readable => Crash
Now, you can compile with higher optimizations: -O3, example output, stripped again:
main:
jmp main
It is just an infinite loop. Theoretically both would have the same result. The compiler is nearly allowed to do anything, as long as the observable output stays the same (See: https://stackoverflow.com/a/46455917/13912132).
But in the first, it is translated "literally", leading to hitting other parts. But if no other parts would be hit, it would run all the time.
In the second, the compiler may see, that this would run infinite theoretically, so it got optimized.