How to give a pattern for new line in grep?

Question

How to give a pattern for new line in grep? New line at beginning, new line at end. Not the regular expression way. Something like \n.

It's not clear what you want. `new line at the beginning` is a blank line and `new line at end` applies to every line in the file. Can you post an example? — P.P, Sep 29 '12 at 12:11
Actually you can just use `$`. It's somewhat limited, but usable in simple cases. — Krzysztof Jabłoński, Feb 13 '14 at 07:03
https://www.gnu.org/software/grep/manual/grep.html#Usage see item 15. — jarno, Nov 12 '16 at 12:29

score 111 · Answer 1 · answered Oct 01 '12 at 17:46

111

try pcregrep instead of regular grep:

pcregrep -M "pattern1.*\n.*pattern2" filename

the -M option allows it to match across multiple lines, so you can search for newlines as \n.

answered Oct 01 '12 at 17:46

nullrevolution

3,937
1
18
20

This is an excellent suggestion. I've used `pcregrep` a LOT the last week. For example, `pcregrep -lMr "\r\n" *` to recursively find all files that have at least one CRLF line ending. And after conversion to Unix line endings, I used `pcregrep -lMr " \n" *` to find all files having Markdown *soft* line breaks. Very helpful! – Henke Mar 10 '23 at 16:33

score 105 · Accepted Answer · edited Oct 20 '14 at 23:37

105

grep patterns are matched against individual lines so there is no way for a pattern to match a newline found in the input.

However you can find empty lines like this:

grep '^$' file
grep '^[[:space:]]*$' file # include white spaces

edited Oct 20 '14 at 23:37

davidg

5,868
2
33
51

answered Sep 29 '12 at 12:23

arash kordi

2,470
1
22
24

5

At least GNU grep has -z option that makes grep break lines by null character. However, it does not seem to support newline or \n in pattern to match newline, see [bug report](https://bugs.launchpad.net/ubuntu/+source/grep/+bug/1641092). – jarno Nov 12 '16 at 12:15
2

@jarno It supports \n in patterns with the -P flag – Johannes Riecken Mar 09 '18 at 11:01

score 38 · Answer 3 · edited Mar 09 '22 at 08:34

38

Thanks to @jarno I know about the -z option and I found out that when using GNU grep with the -P option, matching against \n is possible. :)

Example:

grep -zoP 'foo\n\K.*'<<<$'foo\nbar'

Result:

bar

Example that involves matching everything including newlines:

.* will not match newlines. To match everything including newlines, use¹ (.|\n)*:

grep -zoP 'foo\n\K(.|\n)*'<<<$'foo\nbar\nqux'

Result:

bar
qux

¹ Seen here: https://stackoverflow.com/a/33418344

edited Mar 09 '22 at 08:34

Robin A. Meade

1,946
18
17

answered Mar 09 '18 at 11:10

Johannes Riecken

2,301
16
17

1

Great, note that I do not get notification about the "@jarno" in the answer. – jarno Mar 13 '18 at 11:59
3

This is great! Thank you so much for sharing! – Alicia Tang Apr 06 '18 at 17:43
Works for me. Thanks a lot. `grep -zoPnir "dex\n035" ./dir/` – galian Dec 18 '20 at 07:11

score 26 · Answer 4 · edited Oct 18 '15 at 11:52

26

You can use this way...

grep -P '^\s$' file

-P is used for Perl regular expressions (an extension to POSIX grep).
\s match the white space characters; if followed by *, it matches an empty line also.
^ matches the beginning of the line. $ matches the end of the line.

edited Oct 18 '15 at 11:52

kenorb

155,785
88
678
743

answered Sep 29 '12 at 12:30

Manikandan Rajendran

1,142
1
8
9

2

`-P` is a GNU extension. I am fine for using it when the situation calls for it (typically lookahead/lookbehind), but POSIX grep can do this just file with `[[:space:]]`. – jordanm Sep 29 '12 at 21:29
1

FWIW, Solaris' and BSD's grep manpages (didn't check others) both have a paragraph for `-P`. GNU is quite standard anyway. :) – K3---rnc Jun 24 '13 at 05:09
1

From manual page: [-P] This option is not supported in FreeBSD. – Marián Černý Dec 04 '14 at 13:31
No [-P] for MacOsX either: grep [-abcdDEFGHhIiJLlmnOopqRSsUVvwxZ] [-A num] [-B num] [-C[num]] [-e pattern] [-f file] [--binary-files=value] [--color[=when]] [--colour[=when]] [--context[=num]] [--label] [--line-buffered] [--null] [pattern] [file ...] – Niccolò Jan 12 '17 at 10:50
@Niccolò You'll want to `brew install grep` to get GNU grep, which is superior in several ways. https://apple.stackexchange.com/questions/193288/how-to-install-and-use-gnu-grep-in-osx – Ethan Herdrick May 11 '18 at 23:21

kenorb · Answer 5 · 2018-05-13T23:56:23.583

8

As for the workaround (without using non-portable -P), you can temporary replace a new-line character with the different one and change it back, e.g.:

grep -o "_foo_" <(paste -sd_ file) | tr -d '_'

Basically it's looking for exact match _foo_ where _ means \n (so __ = \n\n). You don't have to translate it back by tr '_' '\n', as each pattern would be printed in the new line anyway, so removing _ is enough.

edited May 13 '18 at 23:56

answered Oct 18 '15 at 12:04

kenorb

155,785
88
678
743

1

pardon my ignorance, but is using process substituion not yet another NON posix compatible, hence non-portable feature? in posix shell I could not do `<(p paste -sd_ file )`? – humanityANDpeace Sep 06 '18 at 11:23

score 6 · Answer 6 · answered Dec 29 '18 at 10:21

6

just found

grep $'\r'

It's using $'\r' for c-style escape in Bash.

in this article

answered Dec 29 '18 at 10:21

HHHartmann

61
1
1

carriage return `\r` is not a newline `\n`. If your grepping for windows style line ending you need `\r\n` – CervEd Jan 07 '22 at 11:27

How to give a pattern for new line in grep?

6 Answers6

Linked

Related