1

When I try to divide, 1234567890 (10 digits; less than max unsigned int), I don't get int overflow When I try to divide, 2345678901 (10 digits; less than max unsigned int), I get int overflow.

When I divide the first number, EDX is filled with 0's, then I try the second number, EDX is filled with 1's.

An observation is that when I try to enter 2345678901 into my Windows programmer calculator set to DWORD, it won't let me enter the last digit (1). So I can enter 234567890.

What is another approach I can take? Not sure what else I need to do. I thought about zeroing out EAX, but that seems pointless when I am just going to mov the value into eax

    xor edx, edx
; converts the string
        NextDigit:
            mov ebx, 10
            cdq
            div ebx
            add edx, 48         ; add to remainder
            push    eax         ; save eax (quotient)
            mov eax, edx        ; remainder + 48 into eax

The error:

Picture of Error

Michael Petch
  • 46,082
  • 8
  • 107
  • 198
Glenn Oberlander
  • 53
  • 6
  • 3
    DIV is *unsigned* divide.If you sign extend EAX into EDX with `CDQ` then you will get a large 64-bit number if EAX is >= 0x80000000 (sign bit set). That is the case with the value 2345678901 (0x08BD03835 where the top bit is set). After `CDQ` EDX will have 0xFFFFFFFF and EAX will have 0x08BD03835).That is the unsigned number 18446744071760263221 . Divide that by 10 and you get a big number that can't fit in EAX so it is division overflow. You can use unsigned division to do this IF you set EDX to 0 instead of using CDQ. Replace `CDQ` with `xor edx, edx` . – Michael Petch Mar 12 '19 at 21:12
  • 1
    Thank you. I tried that and it worked. – Glenn Oberlander Mar 12 '19 at 23:04

0 Answers0