How can I delete all lines before a specific string from a number of files

Question

I have n files, like:

file1:

1aaa
2eee

Test        XXX
Hanna
Lars

file2:

1fff
2ddd
3zzz

Test        XXX
Mike
Charly

I want to remove all rows before "Test XXX" from all n files. The number of rows to delete varies between files.

My idea:

for file in 1 :n
do
pos=grep -n "Test XXX" file$file
sed -i "1:$pos-1 d" file$file >new$file
done

seems a duplicate of http://stackoverflow.com/questions/5935742/how-to-ignore-all-lines-before-a-match-occurs-in-bash — ErichBSchulz, Mar 03 '13 at 03:54

score 28 · Answer 1 · answered Sep 14 '11 at 13:37

28

This should work for you:

sed -i '1,/Test XXX/d' file1
sed -i '1,/Test XXX/d' file2

or simply

sed -i '1,/Test XXX/d' file*

answered Sep 14 '11 at 13:37

ztank1013

3

Be careful with this you will lose the original file. – Lynch Sep 15 '11 at 02:06
1

That is exactly the purpose of the -i (inline) option. – ztank1013 Sep 15 '11 at 05:45
7

Just use `-i.bak` if you want to create a backup file at the same time. – l0b0 Sep 15 '11 at 14:42
sed '1,/Test XXX/d' < file > newfile – ErichBSchulz Mar 03 '13 at 03:52
This seems to fail if the matched line is the first line. – Alec Jacobson Aug 27 '13 at 10:05
7

This seems to be answering a different question; the question was how to delete lines BEFORE a specific line, not including. – Tim Harper Jan 15 '16 at 00:02
how can i remove all rows after some string? – colintobing May 20 '17 at 22:37
1

This, unfortunately, deletes the Test XXX line as well... : / – Jewenile Aug 22 '17 at 08:05
Be careful if you use this in shell script , if you use variable for sed argument, replace single quote with double qoute – Scott Chu May 31 '18 at 09:42
pro to @Jewenile, it eats first match line. Alec Jacobson's answer solves this problem. – Scott Chu May 31 '18 at 09:57

score 10 · Answer 2 · answered Aug 27 '13 at 11:48

10

This will work for your examples and even if the matched pattern is on the very first line:

sed -n -E -e '/Text XXX/,$ p' input.txt | sed '1 d'

For example if you input is simply

Test        XXX
Mike
Charly

This will give you

Mike
Charly

If you want to keep the first match Test XXX then just use:

sed -n -E -e '/Text XXX/,$ p' input.txt

answered Aug 27 '13 at 11:48

Alec Jacobson

1

I find a url talking undocument -E (however it seems a ded link; thanks to Google, it has web-storage version). It's eqivalent with -r, which for compatible with BSD ( see the code segment in that dead link page: `/* Undocumented, for compatibility with BSD sed. */ case 'E': case 'r': if (extended_regexp_flags) usage(4); extended_regexp_flags = REG_EXTENDED; break; . . .` – Scott Chu Jun 01 '18 at 03:06
That link is sometimes ok, so I'll post its url here: http://blog.dmitryleskov.com/small-hacks/mysterious-gnu-sed-option-e/ – Scott Chu Jun 01 '18 at 03:16
From the manual (`man sed`): `-E, -r, --regexp-extended` ; `use extended regular expressions in the script (for portability use POSIX -E).` Also https://www.gnu.org/software/sed/manual/html_node/Extended-regexps.html – Wilf Mar 25 '19 at 22:03

Tim Harper · Answer 3 · 2016-01-15T02:08:03.707

cat <<-EOF > file1.txt
1aaa
2eee

Test        XXX
Hanna
Lars
EOF

cat file1.txt | sed -e '/Test *XXX/p' -e '0,/Test *XXX/d'

Output:

Test        XXX
Hanna
Lars

Explanation:

-e '/Test *XXX/p' duplicates the line matching /Test *XXX/
-e '0,/Test *XXX/d' deletes from line 0 to the first line matching /Test *XXX/

By duplicating the line, then removing the first one, we effectively retain the matched line, successfully deleting all lines BEFORE Test XXX

Note: this will not work as expected if there are multiple Test XXX lines.

score 1 · Answer 4 · answered Sep 14 '11 at 13:25

1

You can do it with bash ( eg for 1 file)

t=0
while read -r line
do
    [[ $line =~ Test.*XXX ]] && t="1"
    case "$t" in
     1) echo "$line";;
    esac
done < file > tempo && mv tempo file

Use a for loop as necessary to go through all the files

answered Sep 14 '11 at 13:25

bash-o-logist

4 Answers4