1

I have a log file which contains several repeats of the pattern Fre --. I need to remove only first occurrence of this pattern and the next 20 lines after that and keep other matches intact. I need to do it in a bash terminal, using sed preferably or awk or perl. I would highly appreciate your help.

I tried 

sed -e '/Fre --/,+20d' log.log

but it deletes all the patterns and next 20 lines after that. I want only first pattern to be removed

There is a more or less similar question and some answers here: How to remove only the first occurrence of a line in a file using sed but I don't know how to change it to remove 20 lines after the first match

Sundeep
  • 23,246
  • 2
  • 28
  • 103
Amin Alibakhshi
  • 25
  • 2

2 Answers2

2

Pretty sure that someone will find a nice sed command but I know awk better.

You can try :

awk '/Fre --/ && !found++{counter=21}--counter<0' log.log

Explanations :

/Fre --/ -> if it finds pattern Fre --

&& !found++ -> and if it didn't find it before

{counter=21} -> it sets counter value at 21 (because you want to remove the line + the next 20s)

--counter<0 -> decreases the counter and prints the line only if counter < 0

As mentioned by @Sundeep, @EdMorton solution is safer on very big files.

awk '/Fre --/ && !found++{counter=21}!(counter&&counter--)' log.log

NOTE

If you want the deletions to be saved into the original file, you will have to copy the contents of the awk command into a temp file, and then move the temp file into the original file. Always be careful before editing the original file since you may lose precious informations.

Run the first command first :

awk '/Fre --/ && !found++{counter=21}!(counter&&counter--)' log.log > log.log.tmp

Then check the .tmp file and you can run the second command to apply the changes if .tmp file looks ok :

mv log.log.tmp log.log
Corentin Limier
  • 4,946
  • 1
  • 13
  • 24
  • 2
    With `GNU sed`, it would be something like `sed '0,/Fre --/{//{N;N;N;N;N;N;N;N;N;N;N;N;N;N;N;N;N;N;N;N;d}}'` .. `awk` is better suited here.. one thing though, your counter could cause issue if the input file is large enough due to finite precision – Sundeep Oct 17 '19 at 14:39
  • 2
    There may be a sed command but it definitely won't be nice :-). – Ed Morton Oct 17 '19 at 14:47
  • @EdMorton @Sundeep > I tried to combine those two commands and couldn't find a way. https://stackoverflow.com/questions/35700633/sed-remove-first-occurence-of-string-from-file https://stackoverflow.com/questions/4396974/sed-or-awk-delete-n-lines-following-a-pattern . Something like `sed '0,/test3/{/test3/+3d}' file` . I'm surprised that such a solution does not exist. – Corentin Limier Oct 18 '19 at 09:54
  • 1
    Thanks @RaphaelSetin for adding the info on how to actually delete the lines on the original file. I updated a bit your suggestions since I prefer to avoid showing commands that could delete information on original files on stackoverflow. Developers sometimes copy/paste commands and could lose precious informations. – Corentin Limier Apr 29 '21 at 07:27
1
$ seq 20 | awk '!f && /3/{c=4; f=1} !(c&&c--)'
1
2
7
8
9
10
11
12
13
14
15
16
17
18
19
20

See Printing with sed or awk a line following a matching pattern

Ed Morton
  • 188,023
  • 17
  • 78
  • 185
  • 1
    I was trying something similar from that Q&A (I've bookmarked the link)... I got `awk 'c&&c--{next} /3/&&!f{f=1; c=3; next} 1'` before seeing your optimized version – Sundeep Oct 17 '19 at 14:43
  • 1
    It could be made slightly more brief too as `awk '/3/&&!f++{c=4}!(c&&c--)'` but we're trading off clarity a bit too much now IMHO! – Ed Morton Oct 17 '19 at 14:45
  • 1
    @AminAlibakhshi safer solution than mine on big files – Corentin Limier Oct 18 '19 at 09:41