Assembly - Round floating point number to .001 precision toward -∞

Question

I am trying to write all my floating point numbers to .001 precision toward -∞. I have set the RC field to 01 binary, but I am only able to print out the initial floating point number with the wanted precision and thereafter it disregards the rounding. I think I might be missing something obvious in how I am handling the precision towards -∞, but I am not sure.

INCLUDE Irvine32.inc

.data

ctrlWord  WORD  010000000000b ; set the RC field to round down toward -∞.

.code

     fild   sum                         ; load integer into ST(0)
     fidiv  count                       ; divide ST(0) by mem int
     fstcw  ctrlWord                    ; store control word
     fldcw  ctrlWord                    ; load control word
     call   WriteFloat                  ; +2.5000000E+000

     fild   stdDev                      ; load variance integer into ST(0)
     fidiv  count                       ; average the result 
     fsqrt                  ; ST(0) = square root
     call   WriteFloat                  ; +2.5495097E+000

'numbers to .001 precision toward': Do you mean decimal or binary? — rkhb, Apr 29 '14 at 08:07

rkhb · Accepted Answer · 2015-12-17T12:07:52.067

The rounding mode of the FPU applies only to the last binary digit of the floating point mantissa. It cannot be used for a decimal rounding. There is one exception from that rounding rule: FRNDINT. This instruction "rounds the source value in the ST(0) register to the nearest integral value, depending on the current rounding mode (setting of the RC field of the FPU control word), and stores the result in ST(0)." (Intel Manual).

For decimal rounding you can use FRNDINT with a trick: First calculate n * 10^x (x is the desired decimal precision of the fractional part). Then use FRNDINT which rounds the integral part and truncates the fractional part. Finally you can divide the number by 10^x or you can convert the number to a string and insert the decimal point appropropriate.

Take a look at this example (it's your code, just a little bit modified):

INCLUDE Irvine32.inc

.data

newCtrlWord  WORD 010000000000b ; set the RC field to round down toward -∞.
oldCtrlWord  WORD ?
sum WORD 25
count WORD 10
stdDev WORD 65
thousand WORD 1000

.code
main:

    fstcw oldCtrlWord               ; store control word
    mov ax, oldCtrlWord
    and ah, 11110011b               ; clear _only_ RC field
    or  ah, 00000100b               ; set _only_ RC field (rounding to -∞)
    mov newCtrlWord, ax
    fldcw newCtrlWord               ; load control word

    fild   sum                      ; load integer into ST(0)
    fidiv  count                    ; divide ST(0) by mem int
    fimul thousand                  ; thousand=10^3 - preserve three decimal digits of the fractional part
    frndint                         ; only for this you need the control word
    fidiv thousand                  ; adjust integer to a correct rational number (reverse the `fimul thousand`)
    call WriteFloat                 ; +2.5000000E+000
    call CRLF

    fild stdDev                     ; load variance integer into ST(0)
    fidiv count                     ; average the result
    fsqrt                           ; +2.5495097E+000
    fimul thousand                  ; thousand=10^3 - preserve three decimal digits of the fractional part
    frndint                         ; only for this you need the control word
    fidiv thousand                  ; adjust integer to a correct rational number (reverse the `fimul thousand`)
    call WriteFloat                 ; +2.5490000E+000
    call CRLF

    fldcw  oldCtrlWord              ; restore control word

    exit

END main

Assembly - Round floating point number to .001 precision toward -∞

1 Answers1