231

I want to grep the shortest match and the pattern should be something like:

<car ... model=BMW ...>
...
...
...
</car>

... means any character and the input is multiple lines.

kenorb
  • 155,785
  • 88
  • 678
  • 743
syker
  • 10,912
  • 16
  • 56
  • 68

7 Answers7

377

You're looking for a non-greedy (or lazy) match. To get a non-greedy match in regular expressions you need to use the modifier ? after the quantifier. For example you can change .* to .*?.

By default grep doesn't support non-greedy modifiers, but you can use grep -P to use the Perl syntax.

Mark Byers
  • 811,555
  • 193
  • 1,581
  • 1,452
  • "You will also need the dot all modifier so that the dot matches new lines." This answer is the top result for "grep dot all modifier" ... what is it? – jameshfisher Sep 15 '11 at 10:22
  • 3
    eegg: dot all modifier is also known as multiline. It's a modifier that changes the "." match behavior to include newlines (normally it doesn't). There's no such modifier in grep, but there is in [pcregrep](http://linuxcommand.gds.tuwien.ac.at/man_pages/pcregrep1.html). – A. Wilson May 07 '12 at 22:01
  • 2
    Correction: In most of the regex flavors that support it, the mode that allows `.` to match newlines is called *DOTALL* or *single-line* mode; Ruby is the only one that calls it *multiline*. In the other flavors, *multiline* is the mode that allows the anchors (`^` and `$`) to match at line boundaries. Ruby has no equivalent mode because in Ruby they always work that way. – Alan Moore Sep 08 '12 at 21:40
  • 7
    `-P` was a complete new one on me, I've been happily grepping away for years, and only using `-E` ... so many wasted years! - Note to self: Re-read Man pages as a (even more!) regular thing, you never digest enough switches and options. – ocodo Aug 15 '13 at 02:43
  • 40
    On some platforms (like Mac OS X) `grep` does not support `-P`, but if you use `egrep` you can use the `.*?` pattern to achieve the same result. `egrep -o 'start.*?end' text.html` – SaltyNuts Feb 21 '14 at 16:05
  • 7
    As an extension to @SaltyNuts comment, Mac OS X does not support `-P` but `-E` would call `egrep` hence the suggested `.*?` works just fine. – Fredrik Erlandsson Dec 15 '14 at 07:12
  • This answer and comments have so much useful info. Thanks. – Ulises Layera Feb 12 '15 at 19:16
  • I can't tell the difference of using `.*` or `.*?`, until I use the `-o` option – cifer Apr 01 '16 at 07:07
  • `man grep` says: `This is highly experimental and grep -P may warn of unimplemented features.` Why not use Perl itself? Something like `perl -ne 'print if /match/'` – yaeuge May 02 '17 at 10:59
  • 1
    This worked perfectly on OSX but sadly doesn't (fails silently - continues to greedy match) on alpine linux alpine:3.7. Does anyone know of an alternative? – David Aug 21 '18 at 16:10
  • I use MacOS. I have both `grep` and `ggrep` (ggrep installed with Homebrew). By running these 4 commands, you can see why I prefer using `ggrep -P` vs `grep -E`. `echo "part_1:part_2:" | grep -E '^.*?:' --color=always`, `echo "part_1:part_2:" | grep -E -o '^.*?:' --color=always`, `echo "part_1:part_2:" | ggrep -P '^.*?:' --color=always`, `echo "part_1:part_2:" | ggrep -P -o '^.*?:' --color=always`. – Touten Jul 09 '20 at 22:38
96

Actualy the .*? only works in perl. I am not sure what the equivalent grep extended regexp syntax would be. Fortunately you can use perl syntax with grep so grep -P would work but grep -E which is same as egrep would not work (it would be greedy).

See also: http://blog.vinceliu.com/2008/02/non-greedy-regular-expression-matching.html

kenorb
  • 155,785
  • 88
  • 678
  • 743
John Smith
  • 961
  • 1
  • 6
  • 3
  • 10
    `grep -P` does not work in GNU grep 2.9 -- just tried it (it doesnt error, just silently doesn't apply the `?`. Intertestly neither does the *not class* eg: `env|grep '[^\=]*\='` – roberto tomás Oct 24 '11 at 22:56
  • 2
    There's no `grep -P` option or `pgrep` command in Darwin/OS X 10.8 Mountain Lion, but `egrep` works great. – Steve HHH Aug 16 '13 at 19:13
  • 2
    There's a `pgrep` command on my OS X 10.9 box, but it's a completely different program whose purpose is to "find or signal processes by name". – Desty Jul 11 '14 at 13:23
  • @robertotomás Responding to a 6-year old comment here, but....I thought this as well and then realized I was getting multiple non-greedy matches. For instance, on a color terminal you can see that ` echo "bbbbb" | grep -P 'b.*?b'` returns 2 matches. – zzxyz Nov 01 '17 at 00:27
  • In 2023 on macOS Monterey 12.6 `grep -E` worked, made it so it uses the perl regex format. – Nikolay Suvandzhiev Jan 05 '23 at 21:39
23

grep

For non-greedy match in grep you could use a negated character class. In other words, try to avoid wildcards.

For example, to fetch all links to jpeg files from the page content, you'd use:

grep -o '"[^" ]\+.jpg"'

To deal with multiple line, pipe the input through xargs first. For performance, use ripgrep.

kenorb
  • 155,785
  • 88
  • 678
  • 743
22

My grep that works after trying out stuff in this thread:

echo "hi how are you " | grep -shoP ".*? "

Just make sure you append a space to each one of your lines

(Mine was a line by line search to spit out words)

Jan Turoň
  • 31,451
  • 23
  • 125
  • 169
jonz
  • 221
  • 2
  • 2
6

Sorry I am 9 years late, but this might work for the viewers in 2020.

So suppose you have a line like "Hello my name is Jello". Now you want to find the words that start with 'H' and end with 'o', with any number of characters in between. And we don't want lines we just want words. So for that we can use the expression:

grep "H[^ ]*o" file

This will return all the words. The way this works is that: It will allow all the characters instead of space character in between, this way we can avoid multiple words in the same line.

Now you can replace the space character with any other character you want. Suppose the initial line was "Hello-my-name-is-Jello", then you can get words using the expression:

grep "H[^-]*o" file
colidyre
  • 4,170
  • 12
  • 37
  • 53
mr.1n5an_e
  • 79
  • 1
  • 3
1

The short answer is using the next regular expression:

(?s)<car .*? model=BMW .*?>.*?</car>
  • (?s) - this makes a match across multiline
  • .*? - matches any character, a number of times in a lazy way (minimal match)

A (little) more complicated answer is:

(?s)<([a-z\-_0-9]+?) .*? model=BMW .*?>.*?</\1>

This will makes possible to match car1 and car2 in the following text

<car1 ... model=BMW ...>
...
...
...
</car1>
<car2 ... model=BMW ...>
...
...
...
</car2>
  • (..) represents a capturing group
  • \1 in this context matches the sametext as most recently matched by capturing group number 1
jmc
  • 47
  • 4
-2

I know that its a bit of a dead post but I just noticed that this works. It removed both clean-up and cleanup from my output. 

> grep -v -e 'clean\-\?up'
> grep --version grep (GNU grep) 2.20
user200850
  • 97
  • 1