1

I've been using the below hexdump format string on Linux for displaying the contents of a binary file in C-style hex format \xXX\xXX...:

$ hexdump -v -e '1/1 "\\x%02x"' file.bin
hexdump: %?: bad conversion character

However, as can be seen above, the \ in\\ isn't escaped properly, with the result of an error.

How can I fix this?

Shuzheng
  • 11,288
  • 20
  • 88
  • 186

1 Answers1

4

Hexdump Issue is Macos-Only

This problem used to be a hexdump bug on debian, but was fixed. I've tested this on FreeBSD, and it's fixed in the October 29, 2014 version of hexdump.

The version of BSD hexdump on Macos (see man hexdump) has not been updated since July 10, 2004 and still has this problem. For more info on trying to find the version of Macos utilities, this SO question regarding sed is helpful.

hexdump

We can get around it by putting the \x in a separate format string. For more information on why this is necessary, this SO question about escaping tabs is useful. 

hexdump -v -e '"\\\x" 1/1 "%02x"' $file

Three backslashes are necessary to get a \ literal with x following.

xxd

For sake of completeness, an xxd implementation takes more steps, but doesn't require formatting string shenanigans:

xxd -p $file | tr -d '\n' | sed 's/\(..\)/\\x\1/g'
Ross Jacobs
  • 2,962
  • 1
  • 17
  • 27
  • Thanks, so it's a bug in macOS `hexdump` only? Wouldn't it be better to also but `1/1` in front of `\\\x`, or should the byte count be 0, so that format doesn't consume any bytes (I'm not sure)? – Shuzheng Sep 19 '19 at 05:05
  • This bug applies to all forms of hexdump. The linked SO question has more information about the bug, which is [this Debian one](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=776096). Edited answer to link directly to the bug. – Ross Jacobs Sep 19 '19 at 05:39
  • If you put 1/1 in front of \\\x, then a byte will be read and won't be added as '\\\x' is a string literal. So that byte will be dropped, and if you put 1/1 in front of \\\x, every other byte won't be output by hexdump – Ross Jacobs Sep 19 '19 at 05:43
  • @Shuzheng if you think this is the correct answer, please mark it as such. – Ross Jacobs Sep 19 '19 at 22:53
  • I just want to correct you that this bug is not present on current Debian 8.3.0: `echo -n ABCD | hexdump -v -e '1/1 "\\x%02x"'` results in `\x41\x42\x43\x44`. – Shuzheng Sep 20 '19 at 06:47
  • Thanks for the catch. I've added more context. It looks like this is a problem with Macos hexdump, but the BSD and Linux versions have been updated in their respective operating systems. – Ross Jacobs Sep 20 '19 at 17:34
  • Do you know how to update the macOS version? Or where to download code to update it myself? I can't find any update using Brew. – Shuzheng Sep 20 '19 at 18:37
  • @Shuzheng I would just use xxd, tr, and sed instead as shown above. If you need an answer, this would be a good question for https://apple.stackexchange.com. – Ross Jacobs Sep 20 '19 at 18:42