5

I have C function that does some SSE calculations. When I compile it with GCC I get next code

/* Start of function */
mov    (%rdi),%rax
movslq %ecx,%rcx
...
mov    0x8(%rdi),%rax
pxor   %xmm12,%xmm3
movaps %xmm0,-0x28(%rsp)
movaps %xmm6,%xmm1
...
movaps 0x50(%rax,%rcx,1),%xmm2
movaps 0x60(%rax,%rcx,1),%xmm15
pxor   %xmm2,%xmm0
pxor   %xmm2,%xmm6
movaps -0x28(%rsp),%xmm2
pxor   %xmm15,%xmm5
pxor   %xmm15,%xmm2
movaps 0x70(%rax,%rcx,1),%xmm15
movaps (%rax,%rcx,1),%xmm11
mov    0x10(%rdi),%rax
movaps %xmm15,-0x18(%rsp)
pxor   %xmm11,%xmm4
pxor   %xmm12,%xmm11
pxor   %xmm15,%xmm12

Look at movaps instructions - it's access memory over stack top:

movaps %xmm15,-0x18(%rsp)

Isn't it an access to undefined memory? And why GCC generated such incorrect code?

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
Alexander Dzyoba
  • 4,009
  • 1
  • 24
  • 29
  • 2
    Are there other instructions in the function (especially in the preamble or various exit points) that adjust `%rsp` in various ways. I might guess that, for a particular block/loop in the function, `%rsp` was adjusted downward to make room for some temporary variables, and at this point, it's already been adjusted back, but `gcc` knows it hasn't overwritten those locations yet, so it's free to still access them. Or maybe creating some temp variables without bothering to adjust `%rsp`... Optimization can do what looks like weird stuff... – twalberg Dec 18 '13 at 15:20
  • @twalberg nope, `%rsp` is not adjusted anywhere neither in this function nor in anything after call. – Alexander Dzyoba Dec 19 '13 at 07:10

1 Answers1

8

There's no such thing as "undefined memory" at the assembly level. gcc is free to emit code that accesses the stack in whatever way it sees fit, so long as the behaviour is as expected.

My guess as to why this is happening is that this is a leaf function for which adjusting the stack pointer is fruitless. You could try to verify that by inspecting the assembly for any call instructions. (You could also inspect the C source, but inlining may make that a bit less reliable.)

This kind of trickery is explicitly allowed by the ABI of certain platforms, including x86-64. From the AMD64 ABI documentation:

The 128-byte area beyond the location pointed to by %rsp is considered to be reserved and shall not be modified by signal or interrupt handlers. Therefore, functions may use this area for temporary data that is not needed across function calls. In particular, leaf functions may use this area for their entire stack frame, rather than adjusting the stack pointer in the prologue and epilogue. This area is known as the red zone.

This blog post might make for interesting reading on the subject.

gsg
  • 9,167
  • 1
  • 21
  • 23
  • It's indeed leaf function. But the real problem is that this is actually kernel module function, so i think memory above `rsp` can be accessed by other threads or kernel itself. That's what bothers me most. – Alexander Dzyoba Dec 19 '13 at 07:12
  • 1
    I see. I don't know anything about the kernel ABI, but there's a clause in the x86-64 ABI to allow accesses within a certain fixed size below the stack pointer, with the goal of allowing this leaf-function optimization. I'll edit in some references... – gsg Dec 19 '13 at 08:19
  • 5
    You were right, thank you so much! The problem was in data corruption because interrupt handler was overwriting stack frame. Quote from [LKD](http://www.makelinux.net/books/lkd2): _Historically, interrupt handlers did not receive their own stacks. Instead, they would share the stack of the process that they interrupted_. So, when I compiled code with `-mno-red-zone` it starts working correct. – Alexander Dzyoba Dec 19 '13 at 08:56
  • 1
    Oh, that sounds like a fun problem to have! Good to hear it is sorted out. – gsg Dec 19 '13 at 09:09
  • @AlexanderDzyoba: yup, [Why can't kernel code use a Red Zone](https://stackoverflow.com/q/25787408) - hardware use of the stack by interrupts. – Peter Cordes Apr 07 '21 at 06:43