12

I've been having some trouble lately with FPU stack overflows. I managed to track it back to a buggy library function that pushes a garbage value onto the FPU stack every time it's called and never cleans it up.

Fortunately, this is easily reproducible and I know exactly what conditions cause it. I can drop a block of inline ASM into the routine that calls this routine to pop the top value back off the FPU stack... except I don't quite know what to write. My ASM-fu is fair to middlin', but not that strong.

So what's the simplest way to get rid of the top value on the FPU stack in x86 assembly, assuming it's garbage data and I don't care about the value?

Mason Wheeler
  • 82,511
  • 50
  • 270
  • 477
  • For the record, x87 is obsolete. Modern code will normally use the XMM registers even for scalar FP math, with instructions like `addss` (scalar single-precision) or [`addsd` (scalar double)](http://felixcloutier.com/x86/ADDSD.html). Look at compiler-generated code for any FP function for examples. – Peter Cordes Aug 17 '18 at 13:40

4 Answers4

14

For Delphi/BASM, in my view the simplest way to pop the FPU stack once is:

asm
 fstp st(0)
end;
PhiS
  • 4,540
  • 25
  • 35
  • 5
    This is actually the correct answer. Most x87 implementations are optimized fot this case and will not even perform the transfer, just pop the value off the stack. – IneQuation Jan 14 '12 at 22:49
10

If you know how much you need to adjust the stack by, you can use fincstp. You also want to ffree the registers that you increment over.

However, probably the simplest solution is to use one of the popping data transfer operations like fstp. Normally you would store the result into an area of memory for later use, something like:

mem_area: defs 10         ; ten bytes for 80 bits
          fstp mem_area   ; pop it

But, if you know you just want to throw away the value, you can use st(0) itself as the destination, saving the memory requirement:

fstp st(0)

See here for a detailed guide on the instructions (particularly this bit).

paxdiablo
  • 854,327
  • 234
  • 1,573
  • 1,953
  • 1
    Note that ' fstp st(0) ' requires one clock. It doesn't get any better than that. – Albert van der Horst Apr 27 '18 at 15:44
  • @AlbertvanderHorst: Superscalar CPUs can easily do better than that. `fstp/fst` with a register destination actually has one per 0.5c throughput (i.e. 2 per clock) on Haswell / Skylake. https://agner.org/optimize/. Register renaming takes care of the dependency through the top-of-stack pointer. – Peter Cordes Aug 17 '18 at 13:43
4

If st0 is the only x87 register in use, you can empty it with:

ffree st0

But this is different from a normal pop if there are multiple stack registers in use, because it doesn't adjust the top-of-stack pointer (TOP field in the x87 status word).

See the registers chapter of the Simply FPU x87 tutorial.

st1 would still be st1 after freeing st0 instead of popping, so this is normally not what you want and has no significant advantage over fstp st0.

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
Jens Björnhager
  • 5,632
  • 3
  • 27
  • 47
  • This doesn't work. st0 is tagged as empty, but that doesn't affect anything except the fpu tag word. – Albert van der Horst Apr 27 '18 at 15:42
  • Not sure what else you would want affected, as it it the definition of what's in the register. – Jens Björnhager Apr 30 '18 at 22:22
  • 1
    Of course you want the stack pointer changed too. I've experimentally verified (by small code words un my Forth, ciforth) that your solution doesn't work as intended. See the answer given by PhiS which is correct. – Albert van der Horst May 09 '18 at 12:29
  • Why would I want the sp changed? The stack is empty, it doesn't matter where you start filling it. – Jens Björnhager May 10 '18 at 01:33
  • Oh I get it, in my answer it says "top" value, it is bad wording. I meant if the top value is the only value on stack. "Bottom" would be correct. – Jens Björnhager May 10 '18 at 01:37
  • If you think the change from top value to bottom value makes your answer correct you should edit that in. I did that for you, but felt obliged to point out a deficiency in your answer. As per my other comment be assured that this is backed up my experimentation. Albert – Albert van der Horst Jul 29 '18 at 12:56
  • I was a bit surprised that someone with your reputation would give an incorrect answer. Now I see that it is literally correct, sorry. If there is only one item on the stack and the stack is otherwise empty, then with setting this one tag word all tag words indicate empty and indeed for this circular stack it doens't matter where the stack pointer points to. I can't see Wheeler indicate that there is only one item on the stack, however. Groetjes Albert – Albert van der Horst Jul 31 '18 at 12:02
  • Yes, the formulation in the answer wasn't clear about what I meant. – Jens Björnhager Aug 17 '18 at 13:19
  • 1
    @JensBjörnhager: your update was technically correct, but still easy to misinterpret for people that didn't know enough about x87 to even think of the only-1-register-in-use condition being relevant / important. I basically rewrote the answer other than the code block to something I think is worth an upvote. – Peter Cordes Aug 17 '18 at 13:37
2

just pop it off the stack with any (fast) instruction that pops. 8087 instruction set

if that doesn't work, FUCOMPP pops twice.

Shaggieh
  • 134
  • 2