1

Can someone explain why 2 different hexa are converted to the same decimal?

$ echo A0000044956EA2 | gawk '{print strtonum("0x" $1)}'
45035997424348832

$ echo A0000044956EA0 | gawk '{print strtonum("0x" $1)}'
45035997424348832
fedorqui
  • 275,237
  • 103
  • 548
  • 598
Eran Ben-Natan
  • 2,515
  • 2
  • 16
  • 19
  • `echo A0000044956EA5 | gawk '{print strtonum("0x" $1)}'` gives `45035997424348840` as output here. Just as a point of information everything between `A0` and `A5` gives the `...32` output though. – Etan Reisner Dec 02 '14 at 13:29
  • @Jidder: I don't think so, as the result is correct (for A0000044956EA0) – Eran Ben-Natan Dec 02 '14 at 13:32
  • @Jidder: Please see `echo a0000044956ea5 | gawk '{print strtonum("0x" $1)}' 45035997424348840`. It seems that after some number, it always convert to octal round number. See for example a0000044956e0a a0000044956e0b a0000044956e0c – Eran Ben-Natan Dec 02 '14 at 14:11
  • @EranBen-Natan i noticed that hence my deleted comments, i think it may be due to rounding now(possibly). –  Dec 02 '14 at 14:20
  • Could this be a floating point precision problem? – Etan Reisner Dec 02 '14 at 14:22
  • 2
    You may need to pick a language with better big number support: `ruby -e 'puts "A0000044956EA1".to_i(16)'` outputs `45035997424348833` – glenn jackman Dec 02 '14 at 14:28
  • @EtanReisner after looking(glancing over) at the awk code it appears all numbers are saved as doubles as `AWKNUM` –  Dec 02 '14 at 15:06
  • Thanks, @glennjackman. Unfortunately, its too late for this... :-( – Eran Ben-Natan Dec 03 '14 at 06:39
  • Pretty related: [Printing long integers in awk](http://stackoverflow.com/q/8857866/1983854) – fedorqui Jan 05 '15 at 12:22

2 Answers2

1

Starting with GNU awk 4.1 you can use --bignum or -M

$ awk -M 'BEGIN {print 0xA0000044956EA2}'
45035997424348834

$ awk -M 'BEGIN {print 0xA0000044956EA0}'
45035997424348832

§ Command-Line Options

0

Not as much an answer but a workaround to at least not bin the strtonum function completely:

It seems to be the doubles indeed. I found the calculation here : strtonum. Nothing wrong with it.

However if you really need this in some awk you should strip the last digit from the hexa number and manually add that after the strtonum did its calculation on the main part of it.

So 0xA0000044956EA1 , 0xA0000044956EA2 and 0xA0000044956EA"whatever" should all become 0xA0000044956EA0 with a simple regex and then add the "whatever".

Edit* Maybe I should delete this all together as I am even to downgrade this even further. This is not working to satisfaction either, just tried it and I actually can't add a number that small to a number this big i.e. print (45035997424348832 + 4) just comes out as 45035997424348832. So this workaround will have to remain having output like 45035997424348832 + 4 for hexa 0xA0000044956EA4.

Robbert Koppenol
  • 103
  • 6