how to grep a hex data area

Question

I have a hex file, I need to extract a range of it to a text file

From range:

To Range:

I need Output: AC:E4:B5:9A:53:1C

i tried many but it not really correct requirements, Output: Binary file filehex matches

grep "["'\x9f\x87\x6f\x11'"-"'\x9f\x87\x70\x11'"]" filehex > test.txt

hope someone can help me

Answer 1

Use -a to force the text interpretation of the input.

Use -o to only output the matching part.

The expression you used doesn't make much sense. It matches any characters in the set \x9, \x87, \x6f, and then the range \x11-\x9f, etc.

You are rather interested in something that starts with \x9\x87\x6f\x11 and ends in \x9f\x87\x70\x11, and there can be anything in between.

You can use cut to remove the leading and trailing 4 bytes.

grep -oa $'\x9f\x87\x6f\x11.*\x9f\x87\x70\x11' hexfile | cut -b5-21

If you know the length of the string will always be 17 bytes, you can use .\{17\} instead of .*.

Answer 2

Ok I've build randomly one binary $file with your string at a location making hd command to split them.

Note: regarding k314159' comment, I use hd to produce hexdump output similarto CentOS's hexdump tool.

One shoot using sed:

hd $file |sed -e 'N;/ 9f \+\(|.*\n[0-9a-f]\+ \+\|\)87 \+\(|.*\n[0-9a-f]\+ \+\|\)6f \+\(|.*\n[0-9a-f]\+ \+\|\)11 /p;D;'
000161c0  96 7a b2 21 28 f1 b3 32  63 43 93 ff 50 a6 9f 87  |.z.!(..2cC..P...|
000161d0  6f 11 0d 7a a5 a9 81 9e  32 9d fb 71 27 6d 60 f2  |o..z....2..q'm`.|
0002c3a0

Explanation:

• N merge next line in current buffer

• \(|.*\n[0-9a-f]\+ \+\|\) match a | followed by anything and a newline (\n), then immediately an hexadecimal number and a space OR nothing.

• p print current buffer (two lines)

• D Delete upto newline in current buffer, keep last line for next sed loop. The last hexadecimal 00028d2a correspond to the size of my binary $file:

  printf "%x\n" $(stat -c %s $file)
  

Using bash + grep:

printf -v var "\x9f\x87\x6f\x11" 
IFS=: read -r offset _ < <(grep -abo "$var" $file)
hd $file | sed -ne "$((offset/16-1)),+4p"
000161a0  b7 8f 4a 4d ed 89 6c 0b  25 f9 e7 c9 8c 99 6e 23  |..JM..l.%.....n#|
000161b0  3c ba 80 ec 2e 32 dd f3  a4 a2 09 bd 74 bf 66 11  |<....2......t.f.|
000161c0  96 7a b2 21 28 f1 b3 32  63 43 93 ff 50 a6 9f 87  |.z.!(..2cC..P...|
000161d0  6f 11 0d 7a a5 a9 81 9e  32 9d fb 71 27 6d 60 f2  |o..z....2..q'm`.|
000161e0  15 86 c2 bd 11 d0 08 90  c4 84 b9 80 04 4e 17 f1  |.............N..|

Where you could read your string:

000161c0                                             9f 87  |              ..|
000161d0  6f 11                                             |o.              |

For testing, I've built my test file by:

dd if=/vmlinuz bs=90574 count=1 of=/tmp/testfile
printf '\x9f\x87\x6f\x11' >>/tmp/testfile 
dd if=/vmlinuz bs=90574 count=1 >>/tmp/testfile
file=/tmp/testfile

Answer 3

Use grep to search for the original binary file, not the hex dump. Extending choroba's answer, I think you may have problems with grep trying to interpret your search pattern as UTF-8 or some other encoding. You should temporarily set the environment variable LC_ALL=C for grep to treat each byte individually. Also, you can use the -P option to enable use of lookbehind and lookahead in your pattern. So your command becomes:

LANG=C grep -oaP $'(?<=\x9f\x87\x6f\x11).*(?=\x9f\x87\x70\x11)' binary-file > test.txt

Proof that it works:

$ echo $'BEFORE\x9f\x87\x6f\x11AC:E4:B5:9A:53:1C\x9f\x87\x70\x11AFTER' | LANG=C grep -oaP $'(?<=\x9f\x87\x6f\x11).*(?=\x9f\x87\x70\x11)'
AC:E4:B5:9A:53:1C
$