Using sed to delete range of lines except the end pattern

Question

input.txt:

blah1
blah2
start_pattern
blah3
blah4
blah5
end_pattern
blah6
blah7

I would like input.txt to become:

blah1
blah2
end_pattern
blah6
blah7

I tried the following, but it didn't work.

sed -i "/start_pattern/, /end_pattern/{$!d}" input.txt

In this question http://stackoverflow.com/q/8164604/356440 they are dealing with multi-line editing and deletion using `sed`, maybe it could give you some clues/tracks. — Stephane Rolland, May 19 '13 at 09:55
In the question I pointed to you, one of the answer links to a guide for `sed` and multi line editing: http://austinmatzko.com/2008/04/26/sed-multi-line-search-and-replace/ this could help. — Stephane Rolland, May 19 '13 at 10:00
sed hasn't been an appropriate tool for multi-line editing since the mid-1970s when awk was invented. Using sed for that is like programming in assembly code but without any benefits. — Ed Morton, May 19 '13 at 12:46

score 9 · Answer 1 · edited May 23 '17 at 11:52

9

This question deals with your problem:

$ sed '/start_pattern/,/end_pattern/{/end_pattern/!d}' foo.txt
blah1
blah2
end_pattern
blah6
blah7

edited May 23 '17 at 11:52

Community

1
1

answered May 19 '13 at 10:11

Adrian Frühwirth

42,970
10
60
71

score 7 · Answer 2 · answered May 19 '13 at 12:36

7

sed is an excellent tool for simple substitutions on a single line. For any other text manipulation just use awk for a clearer, simpler, more easily extensible solution:

$ awk '/start_pattern/{f=1} /end_pattern/{f=0} !f;' file
blah1
blah2
end_pattern
blah6
blah7

See also:

$ awk '/start_pattern/{f=1} !f; /end_pattern/{f=0}' file
blah1
blah2
blah6
blah7

$ awk '!f; /start_pattern/{f=1} /end_pattern/{f=0}' file
blah1
blah2
start_pattern
blah6
blah7

$ awk '/start_pattern/{f=1} /end_pattern/{f=0} f;' file
start_pattern
blah3
blah4
blah5

$ awk '/start_pattern/{f=1} f; /end_pattern/{f=0}' file
start_pattern
blah3
blah4
blah5
end_pattern

$ awk 'f; /start_pattern/{f=1} /end_pattern/{f=0}' file
blah3
blah4
blah5
end_pattern

answered May 19 '13 at 12:36

Ed Morton

188,023
17
78
185

end_pattern might match the same lines as start_pattern, so you should reverse the order of the matches; otherwise this was a great idea, thanks – Matyas Koszik Jun 24 '18 at 16:01
@MatyasKoszik this answer was to the posted question, not to some other question with different input and different requirements. If you'd like to see how to handle data where the start and end regexps both appear on the same line then post a new question with requirements for how you'd like that handled. reversing the order of the match would only help if your input data is populated one of the possible ways it might be populated (end first or start first on the line?) and you have one of the possible requirements for how to handle that. – Ed Morton Jun 24 '18 at 19:13
When I read questions like these I treat the start/end_patterns as placeholders for the actual patterns, which are not specified; as such, they might match the same lines, in which case your solution fails, and with a small improvement, you can handle inputs like those just fine. (BTW, in my scenario start_pattern was "^[osd.$i]" and end_pattern was '^[', and I suspect it's not uncommon that someone wants to delete sections of config files, so by generalizing your solution you're helping them as well. Solution was "/\^[/{f=0} /\^[osd.$id]/{f=1} "\!"f;" ) – Matyas Koszik Jun 28 '18 at 11:20
The **general** case is that the whole file is one single line and the associated general solution is quite different from the solution I posted for this question or the solution you're suggesting in your comment. Even when its just that the start/end strings can appear on 1 line the **general** case would be that there's text between the end string and the start string that you do not want printed so the solution you suggested doesn't work even for that. There's many possibilities and its simply not worth trying to handle them all in one complicated **general** solution when it's not needed. – Ed Morton Jun 28 '18 at 13:08
Something specific to consider - you said `in my scenario start_pattern was "^[osd.$i]" and end_pattern was '^['`. OK, what if in MY scenario `start_pattern was "^[osd.$i]" and end_pattern was '^[' as long as it's not part of "^[osd.$i]"`? Then my script works as intended and your "general" solution fails. The point is it's not general at all it's just what you happen to need for your specific requirements. – Ed Morton Jun 28 '18 at 13:13

Using sed to delete range of lines except the end pattern

2 Answers2