0

I'd like to order the unordered array displayed in the source code. I don't know what I'm missing out here. Ok, I added some more details, hope this helps :)

The program

section .data
    fmt: db "%d",0x0a,0
    arr: dd 1000,100,10,1

section .text
    global main
    extern qsort
    extern printf

main: push   rbp
      mov    rbp, rsp

      mov    rdi, arr
      mov    rsi, 4
      mov    rdx, 4
      mov    rcx, cmp
      call   qsort

      xor    rbx, rbx

.L1:  lea    rdi, [fmt]
      mov    esi, [arr+rbx*4]
      xor    rax, rax
      call   printf
      inc    rbx
      cmp    rbx, 4
      jnz    .L1

      xor    rax, rax
      leave
      ret

cmp:  movsxd rax, dword [rdi]        
      movsxd rbx, dword [rsi]
      sub    rax, rbx
      ret

The program prints

1
100
10
1000
Mikey
  • 35
  • 2
  • Since the output order is not the input order, _something_ has happened. Have you tried just single-stepping through it to see what? You can still debug assembly programs ... – Useless Apr 30 '21 at 17:43
  • I cannot step into qsort() with gdb – Mikey Apr 30 '21 at 17:51
  • 1
    But you could set a breakpoint at `cmp` to see what's going on there. – 1201ProgramAlarm Apr 30 '21 at 17:56
  • What ABI is this, not Win64, [apparently](https://learn.microsoft.com/en-us/cpp/build/x64-calling-convention?view=msvc-160)? Is the assumption that the two arguments to `cmp` are in `rdi` and `rsi` correct? – 500 - Internal Server Error Apr 30 '21 at 18:04
  • In Linux parameters are passed using registers – Mikey Apr 30 '21 at 18:31
  • 1
    `rbx` is a call-preserved register in the Linux x86-64 calling convention. Your functions must leave them with the same value that they found there, but both your `cmp` and `main` modify them without doing so. – Nate Eldredge Apr 30 '21 at 20:00
  • 1
    You also have a bug in `cmp` if `a-b` overflows a 32-bit signed `int`. – Nate Eldredge Apr 30 '21 at 20:04
  • @500-InternalServerError: Yup, we can infer that this is x86-64 System V, which uses different registers from Windows x64, among other differences, used by everything except Windows. [What's the best way to remember the x86-64 System V arg register order?](https://stackoverflow.com/q/63891991) / [Why does Windows64 use a different calling convention from all other OSes on x86-64?](https://stackoverflow.com/a/35619528) – Peter Cordes May 01 '21 at 00:59
  • Note that returning `a-(int64_t)b` like you're doing would be safe and correct *if* `qsort` was expecting a `long` return value, but it's only expecting an `int` (which is 32 bits in x86-64 calling conventions / ABIs). As discussed in comments on [Why do gcc and clang generate mov reg,-1](https://stackoverflow.com/posts/comments/118890049) - AFAIK, if your inputs are the same width as your return value, your best be is probably to return `-1` / `0` / `1` because no single subtraction or other op can cram the 3-way compare-result into the same width :/ – Peter Cordes May 01 '21 at 01:28
  • Note that the subtraction bug won't be a problem for small integers like you're sorting here, only if `a-b` actually has signed overflow when computed as an `int` (in EAX). When EAX == RAX, your code will work after just fixing the register-usage bug. You can create a test case with INT_MIN and INT_MAX to see the other problem Nate spotted. – Peter Cordes May 01 '21 at 02:04

0 Answers0