Bash: delete chars in a file, only on the first line

Question

I am making a script to copy a lot of files from a path to another one. Any files on the path has, on the first line, a lot of "garbage" untill the word "Return-Path....."

File content example:

§°ç§°*é*é*§°ç§°çççççççReturn-PathOTHERTHINGS
REST
OF
THE
FILE
EOF

Probably sed or awk could help on this.

THE PROBLEM:

I want the whole content of the file, except for anything previous then "Return-Path" and it should be stripped ONLY on the first line, in this way:

Return-PathOTHERTHINGS
REST
OF
THE
FILE
EOF

Important thing: anything before Return-Path is "binary", infact files are seen as binary... How to solve?

Mark Setchell · Accepted Answer · 2013-12-06T09:36:28.337

Ok, it's a new day, and now I do feel like coding this for you :-)

The algorithm is described in my other answer to your same question.

#!/bin/bash
################################################################################
# behead.sh
# Mark Setchell
# 
# Utility to remove stuff preceding specified string near start of binary file
#
# Usage: behead.sh <infile> <outfile>
################################################################################
IN=$1
OUT=$2
SEARCH="Return-Path"
for i in {0..80}; do
   str=$(dd if="$1" bs=1 count=${#SEARCH} iseek=$i 2> /dev/null)
   if [ "$str" == $SEARCH ]; then
      # The following line will go faster if you exchange "bs" and "iseek" 
      # parameters, because it will work in bigger blocks, it just looks
      # wrong, so I haven't done it.
      dd if="$1" of="$OUT" bs=1 iseek=$i 2> /dev/null
      exit $?
   fi
done
echo String not found, sorry.
exit 1

You can test it works like this:

#
# Create binary with 15 bytes of bash, then "Return-Path", then entire bash in file "bashed"
(dd if=/bin/bash bs=1 count=15 2>/dev/null; echo -n 'Return-Path'; cat /bin/bash) > bashed
#
# Chop off junk at start of "bashed" and save in "restored"
./behead.sh bashed restored
#
# Check the restored "bash" is exactly 11 bytes longer than original, 
# as it has "Return-Path" at the beginning
ls -l bashed restored

If you save my script as "behead.sh" you will need to make it executable like this:

chmod +x behead.sh

Then you can run it like this:

./behead.sh inputfile outputfile

By the way, there is no concept of "a line" in a binary file, so I have assumed the first 80 characters - you are free to change it, of course!

score 2 · Answer 2 · answered Dec 03 '13 at 19:16

2

Try:

sed '1s/.*Return-Path/Return-Path/'

This command substitutes anything before "Return-Path" with "Return-Path" only on the first line.

answered Dec 03 '13 at 19:16

pfnuesel

14,093
14
58
71

Excuse me, but i forgot specifying that they are "binary" files and they are "binary" till that word... Example of the first two lines: IMS6^@^@^XCP`fL^@^@^@^@^@^@^Xc^@^@^@^@^@^@^@^@^@^@^@^@Return-Path: Received: from asmtp1.website.it (122.1.53.13) by mail1.damn.com – Giuseppe Donato Dec 03 '13 at 19:21

Mark Setchell · Answer 3 · 2013-12-03T19:50:37.280

I don't feel like coding this at this minute, but can give you a hint maybe. "Return-Path" is 11 characters. You can get 11 characters from a file at offset "n" with

dd if=file bs=1 count=11 iseek=n

So if you do a loop with "n" starting at zero and increasing till the result matches "Return-Path" you can calculate how many bytes you need to remove off the front. Then you can do that with another "dd".

Alternatively, have a look at running the file through "xxd", editing that with "sed" and then running it back through "xxd" the other way with "xxd -r".

Bash: delete chars in a file, only on the first line

3 Answers3