Print all the lines between two patterns in shell

Question

I have a file which is the log of a script running in a daily cronjob. The log file looks like-

Aug 19

Line1
Line2
Line3
Line4
Line5
Line6
Line7
Line8
Line9

Aug 19

Aug 20

Line1
Line2
Line3
Line4
Line5
Line6
Line7
Line8
Line9

Aug 20

Aug 21

Line1
Line2
Line3
Line4
Line5
Line6
Line7
Line8
Line9

Aug 21

The log is written by the script starting with the date and ending with the date and in between all the logs are written.

Now when I try to get the logs for a single day using the command below -

sed -n '/Aug 19/,/Aug 19/p' filename

it displays the output as -

Aug 19

Line1
Line2
Line3
Line4
Line5
Line6
Line7
Line8
Line9

Aug 19

But if I try to get the logs of multiple dates, the logs of last day is always missing.

Example- If I run the command

sed -n '/Aug 19/,/Aug 20/p' filename

the output looks like -

Aug 19

Line1
Line2
Line3
Line4
Line5
Line6
Line7
Line8
Line9

Aug 19

Aug 20

I have gone through this site and found some valuable inputs to a similar problem but none of the solutions work for me. The links are Link 1

Link 2

The commands that I have tried are -

awk '/Aug 15/{a=1}/Aug 21/{print;a=0}a'
awk '/Aug 15/,/Aug 21/'
sed -n '/Aug 15/,/Aug 21/p
grep -Pzo "(?s)(Aug 15(.*?)(Aug 21|\Z))"

but none of the commands gives the logs of the last date, all the commands prints till the 1st timestamp as I have shown above.

jantwisted · Answer 1 · 2017-08-22T08:01:40.187

0

You may use multiple patterns by separating with a semicolon.

sed -n '/Aug 19/,/Aug 19/p;/Aug 20/,/Aug 20/p' filename

edited Aug 22 '17 at 08:01

answered Aug 22 '17 at 07:52

jantwisted

1
1

This command only gives for the dates mentioned. If I want the logs for three days this command fails. If I type sed -n '/Aug 19/,/Aug 19/p;/Aug 21/,/Aug 21/p', then the result is for only aug 19 and aug 21, but I need aug 20 also. – Nishant Singh Aug 22 '17 at 08:49
you need to add /Aug 20/,/Aug 20/p too, sed -n '/Aug 19/,/Aug 19/p;/Aug 21/,/Aug 21/p;/Aug 20/,/Aug 20/p' likewise – jantwisted Aug 22 '17 at 09:09

CWLiu · Accepted Answer · 2017-08-24T07:14:17.473

0

I think you can use the awk command as followed to print the lines between Aug 19 & Aug 20,

awk '/Aug 19/||/Aug 20/{a++}a; a==4{a=0}' file

Brief explanation,

/Aug 19/||/Aug 20/: find the record matched Aug 19 or Aug 20
if the criteria is met, set the flag a++
if the flag a in front of the semicolon is greater than 0, that would print the record.
Final criteria, if a==4, then reset a=0, mind that it only worked for the case in the example, if Aug 19 or Aug 20 are more than 4, modify the number 4 in the answer to meet your new request.

If you want to assign the searched patterns into variables, modify the command as followed,

$ b="Aug 19"
$ c="Aug 20"
$ awk -v b="$b" -v c="$c" '$0 ~ c||$0 ~ b{a++}a; a==4{a=0}' file

edited Aug 24 '17 at 07:14

answered Aug 22 '17 at 08:01

CWLiu

3,913
1
10
14

@jantwisted & CWLiu Thanks for such a quick response. – Nishant Singh Aug 22 '17 at 08:46
could you please explain the command as this resolves my issue – Nishant Singh Aug 22 '17 at 08:46
1

@CWLiu: IMHO, it will only look for 4 occurrences of Aug 19, Aug 20. How about if there are more occurrences of Aug 20 etc, it may not print them then. – RavinderSingh13 Aug 22 '17 at 08:47
1

@RavinderSingh13, you're right, the answer only worked for the example in the OP. I will add a note in the last. – CWLiu Aug 22 '17 at 08:50
Thanks for the explanation, and have tried for pulling the logs for multiple dates and it works like a charm. – Nishant Singh Aug 22 '17 at 09:03
Glad it can help :) – CWLiu Aug 22 '17 at 09:04
Hi, can you please help me using the same command with variables, I have tried to modify it without sucess. The command that I tried is "awk -v var="$a" -v vari="$b" '/$var/||/$vari/{a++}a; a==4{a=0}' daily_usage_report.log" in which $a and $b are shell variables. This command does not give any output. – Nishant Singh Aug 23 '17 at 09:46

RavinderSingh13 · Answer 3 · 2017-08-22T08:56:16.193

0

Could you please try following awk solution too once and let me know if this helps you.

awk '/Aug 19/||/Aug 20/{flag=1}; /Aug/ && (!/Aug 19/ && !/Aug 20/){flag=""} flag'  Input_file

EDIT: Adding output too here for letting OP know.

awk '/Aug 19/||/Aug 20/{flag=1}; /Aug/ && (!/Aug 19/ && !/Aug 20/){flag=""} flag' Input_file
Aug 19

Line1
Line2
Line3
Line4
Line5
Line6
Line7
Line8
Line9

Aug 19

Aug 20

Line1
Line2
Line3
Line4
Line5
Line6
Line7
Line8
Line9

Aug 20

edited Aug 22 '17 at 08:56

answered Aug 22 '17 at 08:45

RavinderSingh13

130,504
14
57
93

This command will only print the timestamps but fails to print the logs between them – Nishant Singh Aug 22 '17 at 08:54
@NishantSingh: Kindly see my edit now where I have ran the command and I am getting the output also. Let me know if you have any queries on same. – RavinderSingh13 Aug 22 '17 at 08:56
Tried again on servers running centos 6.9 and ubuntu 14.04. Getting only timestamps as output. – Nishant Singh Aug 22 '17 at 09:07

score 0 · Answer 4 · answered Aug 22 '17 at 09:33

The following approach is quite easy to understand conceptually...

Print all lines from Aug 19 onwards to end of file.
Reverse the order of the lines (with tac because tac is cat backwards).
Print all lines from Aug 21 onwards.
Reverse the order of the lines back to the original order.

sed -ne '/Aug 19/,$p' filename | tac | sed -ne '/Aug 21/,$p' | tac

Print all the lines between two patterns in shell

4 Answers4