Convert binary data to hexadecimal in a shell script

Question

I want to convert binary data to hexadecimal, just that, no fancy formatting and all. hexdump seems too clever, and it "overformats" for me. I want to take x bytes from the /dev/random and pass them on as hexadecimal.

Preferably I'd like to use only standard Linux tools, so that I don't need to install it on every machine (there are many).

Answer 1

Perhaps use xxd:

% xxd -l 16 -p /dev/random
193f6c54814f0576bc27d51ab39081dc

Answer 2

Watch out!

hexdump and xxd give the results in a different endianness!

$ echo -n $'\x12\x34' | xxd -p
1234
$ echo -n $'\x12\x34' | hexdump -e '"%x"'
3412

Simply explained. Big-endian vs. little-endian :D

Answer 3

With od (GNU systems):

$ echo abc | od -A n -v -t x1 | tr -d ' \n'
6162630a

With hexdump (BSD systems):

$ echo abc | hexdump -ve '/1 "%02x"'
6162630a

From Hex dump, od and hexdump:

"Depending on your system type, either or both of these two utilities will be available--BSD systems deprecate od for hexdump, GNU systems the reverse."

Answer 4

All the solutions seem to be hard to remember or too complex. I find using printf the shortest one:

$ printf '%x\n' 256
100

But as noted in comments, this is not what author wants, so to be fair, below is the full answer.

... to use above to output actual binary data stream:

printf '%x\n' $(cat /dev/urandom | head -c 5 | od -An -vtu1)

What it does:

• printf '%x\n' .... - prints a sequence of integers , i.e. printf '%x,' 1 2 3, will print 1,2,3,
• $(...) - this is a way to get output of some shell command and process it
• cat /dev/urandom - it outputs random binary data
• head -c 5 - limits binary data to 5 bytes
• od -An -vtu1 - octal dump command, converts binary to decimal

As a testcase ('a' is 61 hex, 'p' is 70 hex, ...):

$ printf '%x\n' $(echo "apple" | head -c 5 | od -An -vtu1)
61
70
70
6c
65

Or to test individual binary bytes, on input let’s give 61 decimal ('=' char) to produce binary data ('\\x%x' format does it). The above command will correctly output 3d (decimal 61):

$printf '%x\n' $(echo -ne "$(printf '\\x%x' 61)" | head -c 5 | od -An -vtu1)
3d

Answer 5

Perhaps you could write your own small tool in C, and compile it on-the-fly:

int main (void) {
  unsigned char data[1024];
  size_t numread, i;

  while ((numread = read(0, data, 1024)) > 0) {
    for (i = 0; i < numread; i++) {
      printf("%02x ", data[i]);
    }
  }

  return 0;
}

And then feed it from the standard input:

cat /bin/ls | ./a.out

You can even embed this small C program in a shell script using the heredoc syntax.

Answer 6

If you need a large stream (no newlines) you can use tr and xxd (part of Vim) for byte-by-byte conversion.

head -c1024 /dev/urandom | xxd -p | tr -d $'\n'

Or you can use hexdump (POSIX) for word-by-word conversion.

head -c1024 /dev/urandom | hexdump '-e"%x"'

Note that the difference is endianness.

Answer 7

dd + hexdump will also work:

dd bs=1 count=1 if=/dev/urandom 2>/dev/null  | hexdump -e '"%x"'

Answer 8

Sometimes perl5 works better for portability if you target more than one platform. It comes with every Linux distribution and Unix OS. You can often find it in container images where other tools like xxd or hexdump are not available. Here's how to do the same thing in Perl:

$ head -c8 /dev/urandom | perl -0777 -ne 'print unpack "H*"'
5c9ed169dabf33ab

$ echo -n $'\x01\x23\xff' | perl -0777 -ne 'print unpack "H*"'
0123ff

$ echo abc | perl -0777 -ne 'print unpack "H*"'
6162630a

Note that this uses slurp more, which causes Perl to read the entire input into memory, which may be suboptimal when the input is large.

Answer 9

These three commands will print the same (0102030405060708090a0b0c):

n=12
echo "$a" | xxd -l "$n" -p
echo "$a" | od  -N "$n" -An -tx1 | tr -d " \n" ; echo
echo "$a" | hexdump -n "$n" -e '/1 "%02x"'; echo

Given that n=12 and $a is the byte values from 1 to 26:

a="$(printf '%b' "$(printf '\\0%o' {1..26})")"

That could be used to get $n random byte values in each program:

xxd -l "$n" -p                   /dev/urandom
od  -vN "$n" -An -tx1            /dev/urandom | tr -d " \n" ; echo
hexdump -vn "$n" -e '/1 "%02x"'  /dev/urandom ; echo