0

I have an hexadecimal string s and a file f, i need to search the first occurence of that string in the file and save that in a variable with his offset. I thought that the right way to do that is convert the file to hex and search that with a grep. The main problem is that i saw a lot of commands(hexdump,xxd,etc.) to convert but none of them actually work. Any suggestion? My attempt was like this:

xxd -plain $f > $f
grep "$s" .

output should be like:

> offset:filename
SaltyGiampy
  • 85
  • 7
  • 1
    Welcome to Stack Overflow. SO is a question and answer page for professional and enthusiastic programmers. Add your own code to your question. You are expected to show at least the amount of research you have put into solving this question yourself. – Cyrus May 03 '20 at 16:40
  • 1
    Add your file and your desired output to your question (no comment). – Cyrus May 03 '20 at 16:41
  • 1
    Does [Using grep to search for HEX strings in a file](https://stackoverflow.com/questions/6319878/) answer your question? – U880D May 03 '20 at 17:14
  • @U880D I already saw that post and i have really big problems to understand the piping of that one, because i need the file name and the offset, not the content. So it's hard to change that in order to make it that way... – SaltyGiampy May 03 '20 at 18:41

2 Answers2

3

A first approach without any error handling could look like

#!/bin/bash

BINFILE=$1
SEARCHSTRING=$2

HEXSTRING=$(xxd -p ${BINFILE} | tr -d "\n")
echo "${HEXSTRING}"

echo "Searching ${SEARCHSTRING}"
OFFSET=$(grep -aob ${SEARCHSTRING} <<< ${HEXSTRING} | cut -d ":" -f 1)

echo ${OFFSET}:${BINFILE}

I've used xxd here because of Does hexdump respect the endianness of its system?. Please take also note that according How to find a position of a character using grep? grep will return multiple matches, not only the first one. The offset will be counted beginning from 1, not 0. To substract 1 from the variable ${OFFSET} you may use $((${OFFSET}-1)).

I.e. search for the "string" ELF (HEX 454c46) in a system binary will look like

./searchHEX.sh /bin/yes 454c46
7f454c460201010000000000000000000...01000000000000000000000000000000
Searching 454c46
2:/bin/yes
U880D
  • 8,601
  • 6
  • 24
  • 40
  • your method worked really well and i finally understood the xxd, the only problem is that i have to consider the offset in byte, and i saw that there's an option for the cut method instead of d. I'm trying to implement that – SaltyGiampy May 04 '20 at 14:57
  • @SaltyGiampy, thanks for the feedback. Meanwhile I was looking for a solution to have `hexdump` produce the same output as `xxd` ... since that is part of my RHEL basis installations and I would not need to install additional packages. Unfortunately I found an somehow clumsy "re-formatting" approach only and which is `hexdump -C ${BINFILE} | cut -d " " -f 3- | cut -d " " -f -17 | head -n -1 | tr -d " " | tr -d "\n"`. – U880D May 04 '20 at 15:39
  • That looks very tricky but i guess it's a really good solution for your version, that unluckly and weirdly doesn't have xxd in his basic pack... But i can't understand why u delete so many chars with tr or cut and consider only some lines with head – SaltyGiampy May 04 '20 at 17:48
1

I would use regex for this as well:

The text file:

$ cat tst.txt 
1234567890x1fgg0x1cfffrr

A script you can easily change/extend yourself.

#! /bin/bash
part="$(perl -0pe 's/^((?:(?!0(x|X)[0-9a-fA-F]+).)*)(0(x|X)[0-9a-fA-F]+)(.|\n)*/\1:\3\n/g;' tst.txt)"
tmp=${part/:0x*/}
tmp=${#tmp}
echo ${part/*:0x/$tmp:0x} # Echoes 123456789:0x1f

Regex:

^((?:(?!0x[0-9a-fA-F]+).)*) = Search for the first entry that's a hexadecimal number and create a group of it (\1).

(0x[0-9a-fA-F]+) = Make a group of the hexadecimal number (\3).

(.|\n)* = Whatever follows.

Please note that tmp=${part/:0x*/} could cause problems if you have text like :0x before the hexadecimal number that is caught.

Bayou
  • 3,293
  • 1
  • 9
  • 22
  • that would be perfect if the file was already converted to hex, but the file is a regular one, i need to convert bit a bit in hex before the search... – SaltyGiampy May 03 '20 at 20:18
  • 1
    @SaltyGiampy, to [Convert binary data to HEX in a shell script](https://stackoverflow.com/questions/6292645/) you could either use `xxd`, `hexdump` or `od`. – U880D May 04 '20 at 04:05