How to make grep to stop searching in each file after N lines?

Question

It's best to describe the use by a hypothetical example:

Searching for some useful header info in a big collection of email storage (each email in a separate file). e.g. doing stats of top mail client apps used.

Normally if you do grep you can specify -m to stop at first match but let's say an email does not contact X-Mailer or whatever it is we are looking for in a header? It will scan through the whole email. Since most headers are <50 lines performance could be increased by telling grep to search only 50 lines on any file. I could not find a way to do that.

In this particular case, you may be better off with a more specific command like `reformail` to just extract the header(s) you want. See my more detailed [answer here](https://stackoverflow.com/a/64233383/111036) for a similar situation: extracting mail headers. — mivk, Oct 06 '20 at 20:42

rici · Accepted Answer · 2013-07-05T03:23:00.390

I don't know if it would be faster but you could do this with awk:

 awk '/match me/{print;exit}FNR>50{exit}' *.mail

will print the first line matching match me if it appears in the first 50 lines. (If you wanted to print the filename as well, grep style, change print; to print FILENAME ":" $0;)

awk doesn't have any equivalent to grep's -r flag, but if you need to recursively scan directories, you can use find with -exec:

find /base/dir -iname '*.mail' \
     -exec awk '/match me/{print FILENAME ":" $0;exit}FNR>50{exit}' {} +

You could solve this problem by piping head -n50 through grep but that would undoubtedly be slower since you'd have to start two new processes (one head and one grep) for each file. You could do it with just one head and one grep but then you'd lose the ability to stop matching a file as soon as you find the magic line, and it would be awkward to label the lines with the filename.

so far it seems like the most elegant solution. is there a way to make it go recursively through dirs? — Alex, Jul 05 '13 at 03:15

score 1 · Answer 2 · answered Jul 05 '13 at 02:55

1

you can do something like this

head -50 <mailfile>| grep <your keyword>

answered Jul 05 '13 at 02:55

Avichal Badaya

3,423
1
21
23

1

The problem with this approach is that it works only on one file or group of files on fixed number of dir depth. If I want to recursively search for files and grep I would need to add `find` in front and use xargs to pass to `head`. I would also lose filenames or path to file(s) with your solution – Alex Jul 05 '13 at 03:13

kev · Answer 3 · 2013-07-05T05:00:30.273

1

Try this command:

for i in *
do
    head -n 50 $i | grep -H --label=$i pattern
done

output:

1.txt: aaaaaaaa pattern aaaaaaaa
2.txt: bbbb pattern bbbbb

edited Jul 05 '13 at 05:00

answered Jul 05 '13 at 02:56

kev

155,172
47
273
272

Hi kev. What does the sed part do and are you sure it will output file names? – Alex Jul 05 '13 at 03:14
@kev Why using `sed`? `grep -H` can print the filename with matched line. – jaypal singh Jul 05 '13 at 03:19
2

@JS웃: But grep doesn't know the filename; only head knows and it's not telling. You could use `grep "--label=$i"`, though. – rici Jul 05 '13 at 03:25
@rici Sorry didn't see it that way. My bad. Thanks for the explanation though. :) – jaypal singh Jul 05 '13 at 03:48
Number of processes spawned = (2 * n) and OP said: `big collection of email storage (each email in a separate file)` – Amit Naidu May 07 '20 at 05:17

score 0 · Answer 4 · answered Jul 05 '13 at 05:23

0

ls *.txt | xargs head -<N lines>| grep 'your_string'

answered Jul 05 '13 at 05:23

Vijay

65,327
90
227
319

Why not `head -n10 *.txt`. Also, your solution will probably fail on files with spacenames. – Bernhard Jul 05 '13 at 07:05
indeed problem with spaces in names – Alex Jul 05 '13 at 09:18

How to make grep to stop searching in each file after N lines?

4 Answers4

output: