3

I am porting Quake 2's inline assembly code for MSVC to MASM then finally to GAS (for use with MinGW). The specific code in question is for the Skin drawing (R_PolysetCalcGradients for those who want to look it up). The code almost "works" what happens is the skin seems to stretched over the model incorrectly.

Incorrect skin stretching A few interesting things I noticed is when I run objdump -dwrC r_polysa.obj > r_polysa.masm and the same for the GAS version the code is mostly similar except for the fact fsubp and fsubrp have been swapped in MASM. Please note not the operands (I already know about this issue in GAS). Swapped fsubp/fsubrp In the picture the left side is the GAS version, the right side is the MASM version. The original MASM code (and therefore what I have in GAS) is what should be on the left side. I am unsure why MASM is apparently swapping this or if objdump is reporting it incorrectly. However, if I swap the two this does not fix the issue. It just gets inverted in another wrong way. Still wrong skin stretching

I mention this FSUBP/FSUBRP swap because this was a problem in porting the Particle blending inline ASM code. This had one call to use an FSUBRP in the MASM version, objdump reported it now being FSUBP and I had to change it to FSUBP in the GAS version for it to work! I don't understand why this is happening?

In any case, I am new to assembly, but understand some basics and have been doing some reading. Obviously the math here is not quite right, but it seems as though it should be. I don't know how or what to do next. How do I fix and debug this problem?

The code repository to what I am working on is at: https://bitbucket.org/neozeed/q2dos/commits/branch/win32_asm (specifically the Win32_ASM branch). The files I am working with are gas\r_polysa.s and ref_soft\r_polysa.asm.

Maraakate
  • 81
  • 4
  • Why have you reposted your [previous question](https://stackoverflow.com/questions/56210264/objdump-swapping-fsubrp-to-fsubp-on-compiled-assembly)? – fuz May 19 '19 at 19:13
  • That questions was a bit more vague on why the swap is happening. This question shows more of what is going on, with some code and hopefully makes things more clear so someone can see what I am doing wrong. – Maraakate May 19 '19 at 19:15
  • Probably would have been best to just edit the previous question with these details. Maybe you could still find a way to justify talking about (and linking to) the fun project you're working on in that other question :), but I don't think the screenshots are necessary. Seems unlikely that future readers will run into the x87 AT&T syntax design bug with code that's affected the same way, producing similar high-level symptoms. – Peter Cordes May 20 '19 at 05:11
  • No worries, I'll fix it all. This was not intended to be an advertisement for the project. – Maraakate May 20 '19 at 05:27

1 Answers1

1

Opcodes were reversed so there is a quirk with GAS and fsubrp following an fsubp. There was one other additional gotcha, unrelated to that issue. For those interested see: https://bitbucket.org/neozeed/q2dos/commits/f5bf93e3a78e112ae1f766606471a6c5e67283d4

Maraakate
  • 81
  • 4