4

In gdb I'm a bix perplexed as to why sometimes it prints bytes left-to-right and sometimes right-to-left. Here is an example where no instruction has been run in between:

>>> x/4b $rbp-4
0x7fffffffe43c: 0x04    0x00    0x03    0x16
>>> x/2h $rbp-4
0x7fffffffe43c: 0x0004  0x1603

Why is this done in gdb? I would think it would always print them as:

04 00 03 16
Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
samuelbrody1249
  • 4,379
  • 1
  • 15
  • 58

2 Answers2

3

GDB uses the target machine's native endiannes (by default1) to interpret chunks of the size you requested as integers, with addresses increasing from left to right between chunks. x86 is little-endian.

Within a chunk, 04 00 interpreted as a 16-bit little-endian integer is 0x0004. This makes GDB dump an array of short the way you'd expect, for example. You asked GDB for 16-bit integer chunks, so it does what you told it, showing the integer value using standard place-value notation with the leftmost digit the most significant. It's not trying to print the bytes separately because you asked for half-words.

If you want bytes in memory order, use b. That's what it's for.

Footnote 1:

You can change GDB's endianness setting; show endian shows the current setting. However, set endian big causes problems, corrupting register values. e.g. when stopped at _start:

(gdb) p /x $rsp
$1 = 0x7fffffffe6d0       # this is normal for x86-64
(gdb) set endian big
The target is assumed to be big endian
(gdb) x /16xw $rsp
0xd0e6ffffff7f0000:     Cannot access memory at address 0xd0e6ffffff7f0000
(gdb) p /x $rsp
$2 = 0xd0e6ffffff7f0000   # this is obviously broken, byte-reversed

Registers don't have an endianness, and flipping them when expanding their value as an address for another command is just totally broken.

Related not exact duplicates:

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
  • I see, that's pretty tricky! So, since in big endian its stored as `00 04`, it doesn't need to 'flip it' back when it has to interpret 2 bytes correct? – samuelbrody1249 Aug 23 '20 at 23:20
  • 1
    @samuelbrody1249: I wouldn't put it that way. A big-endian interpretation of `00 04` is `0x0004`. For GDB to get that when GDB itself is running on a little-endian host, it will have to byte-swap the value before feeding it to its normal hex-printing function. GDB is printing integers of whatever width your told it to, not separate bytes. – Peter Cordes Aug 23 '20 at 23:26
  • thanks. I've also added an answer myself which is essentially just to type up what I've learned. Let me know if it looks like anything is wrong with my answer if you want to check it out. – samuelbrody1249 Aug 23 '20 at 23:30
1

Peter has the correct and accepted answer but I'll put a very brief explanation here as well that has helped my understanding.

So let’s say we want to store the number 22 (0x16 in hex) as a two-byte value in memory.

Because x86 is little-endian (store most significant byte at highest memory address), we would store it like this:

0xAA1: 0x16                       (LSB)
0xAA2: 0x00                       (MSB)

Now, if we print it byte-by-byte in increasing order of memory address, it obviously looks like this:

0xAA1:  0x16 0x00                (2 bytes, byte by byte in increasing memory address)

But then if we want the original value of 22 back we need to interpret the two-byte value as being little-endian so it returns the correct value:

0xAA1:  0x0016 # 22 in decimal  (bytes flipped back to interpret it properly in program
samuelbrody1249
  • 4,379
  • 1
  • 15
  • 58
  • 1
    Saying the bytes are "flipped back" still sounds to me like the wrong mental model of what's happening. The hex digits of `0x0016` are in place-value order, always most-significant first. The necessary flipping to interpret the 2 bytes as a 16-bit integer with value `22` aka `0x16` happened *before* you got to the stage of printing that value as hex. (Note that GDB internals have to pay extra attention to this because it can do remote debugging. e.g. it could be running on a little-endian x86-64 host and debugging a big-endian MIPS target.) – Peter Cordes Aug 24 '20 at 00:16
  • 1
    But yes, our conventional most-significant-digit first place-value system for writing numbers with arabic numerals happens to be the opposite order of the bytes in memory. So if you're *writing* an int->ASCII hex string function with SIMD on x86 ([like this](//stackoverflow.com/questions/53823756)), you do need a byte-reverse in there somewhere. But one digit at a time, starting with the value in an integer register, you can rotate the top 4 bits to the bottom regardless of memory endianness. Loading it as an integer will already put the binary value in a register you can shift / rotate. – Peter Cordes Aug 24 '20 at 00:21
  • @PeterCordes thanks for the feedback. Wow, that question/answer is a hefty one a bit beyond my current abilities/understanding, but I'll read it over and see what concepts I can pick up. – samuelbrody1249 Aug 24 '20 at 00:26