0

I am essentially trying to use sed to remove a few lines within a text document. To clean it up. But I'm not getting it right at all. Missing something and I have no idea what...

#!/bin/bash

items[0]='X-Received:'
items[1]='Path:'
items[2]='NNTP-Posting-Date:'
items[3]='Organization:'
items[4]='MIME-Version:'
items[5]='References:'
items[6]='In-Reply-To:'
items[7]='Message-ID:'
items[8]='Lines:'
items[9]='X-Trace:'
items[10]='X-Complaints-To:'
items[11]='X-DMCA-Complaints-To:'
items[12]='X-Abuse-and-DMCA-Info:'
items[13]='X-Postfilter:'
items[14]='Bytes:'
items[15]='X-Original-Bytes:'
items[16]='Content-Type:'
items[17]='Content-Transfer-Encoding:'
items[18]='Xref:'

for f in "${items[@]}"; do
    sed '/${f}/d' "$1"
done

What I am thinking, incorrectly it seems, is that I can setup a for loop to check each item in the array that I want removed from the text file. But it's simply not working. Any idea. Sure this is basic and simple and yet I can't figure it out.

Thanks,

Marek

  • 1
    `${f}` won't expand in single quotes. You want double quotes. – FatalError Apr 08 '14 at 04:47
  • 1
    Moreover, you want to _save_ the changes to the file within the loop, else you would end up with the same file. Use the `-i` option for `sed`. – devnull Apr 08 '14 at 04:49
  • why don't you use a `grep -v -f YourSuppressList YourSourceFile` that is faster than sed (a fgrep is still better in your case) ? – NeronLeVelu Apr 08 '14 at 06:12

1 Answers1

2

Much better to create a single sed script, rather than generate 19 small scripts in sequence.

Fortunately, generating a script by joining the array elements is moderately easy in Bash:

regex=$(printf '\|%s' "${items[@]}")
regex=${regex#'\|'}

sed "/^$regex/d" "$1"

(Notice also the addition of ^ to the final regex -- I assume you only want to match at beginning of line.)

Properly, you should not delete any lines from the message body, so the script should leave anything after the first empty line alone:

sed "1,/^\$/!b;/$regex/d" "$1"

Add -i if you want in-place editing of the target file.

Community
  • 1
  • 1
tripleee
  • 175,061
  • 34
  • 275
  • 318
  • In your last line, sed "/^$regex/d" "$1" it should read: sed -i "/^$regex/d" "$1". This works great! Thank you! –  Apr 08 '14 at 05:06
  • Added a version which only touches the headers. This could be important for not mauling MIME multipart messages, for example. – tripleee Apr 08 '14 at 09:59