371

Working with xenserver, and I want to perform a command on each file that is in a directory, grepping some stuff out of the output of the command and appending it in a file.

I'm clear on the command I want to use and how to grep out string(s) as needed.

But what I'm not clear on is how do I have it perform this command on each file, going to the next, until no more files are found.

Patrick
  • 1,635
  • 2
  • 13
  • 23
user2147075
  • 3,711
  • 2
  • 13
  • 3

5 Answers5

391

In Linux, I normally use this command to recursively grep for a particular text within a directory:

grep -rni "string" *

where

  • r = recursive i.e, search subdirectories within the current directory
  • n = to print the line numbers to stdout
  • i = case insensitive search
Philippe Fanaro
  • 6,148
  • 6
  • 38
  • 76
Narain
  • 4,461
  • 2
  • 17
  • 15
345

grep $PATTERN * would be sufficient. By default, grep would skip all subdirectories. However, if you want to grep through them, grep -r $PATTERN * is the case.

umi
  • 3,782
  • 2
  • 15
  • 12
  • 59
    "grep -r $PATTERN ." is sufficient. For recursive grep you do not need to specify all files – Cougar Oct 02 '13 at 06:26
  • 1
    How do I include multiple extensions? Eg. "\*.cpp", "\*.h"? – Tomáš Zato Nov 08 '16 at 16:05
  • 2
    @Tomáš Zato, just supply all your file patterns instead of *: `grep $PATTERN *.cpp *.h`. If you need more specific rules for what files should be grepped, use `find` command (check Rob's answer). – umi Nov 27 '16 at 13:22
  • @umi If I run `grep -r x .` results come from all files of all extensions. If I run `grep -r x *.scss` than I receive no results. – Chris Oct 02 '17 at 21:15
  • 2
    @Chris it's possible you don't have `*.scss` files in current directory but somewhere deeper in subdirs so grep does not look in all the files you wanted. You should use `--include` option to tell grep to look recursively for files that matches specific patterns: `grep -r x --include '*.scss' . ` (note the quotes, they prevent the pattern from being expanded by the shell). Or just use `find` (see Rob's answer). – umi Oct 05 '17 at 19:32
  • 1
    You want `grep -s` so you don't get a warning for each subdirectory that `grep` skips. You should probably double-quote `"$PATTERN"` here. – tripleee Nov 24 '20 at 18:23
50

Use find. Seriously, it is the best way because then you can really see what files it's operating on:

find . -name "*.sql" -exec grep -H "slow" {} \;

Note, the -H is mac-specific, it shows the filename in the results.

Rob
  • 11,446
  • 7
  • 39
  • 57
  • 10
    If you decide to use find, pipe the output through `xargs` rather than using `-exec` this will be *much* faster, because `-exec` spawns a new process for each grep, and the overhead becomes significant with a large number of files. Standard warnings about spaces in file names apply to `xargs`. – Barton Chittenden Feb 04 '15 at 19:50
  • 4
    What would this look like exactly? I always forget `xargs` syntax. – Brian Peterson May 14 '15 at 19:20
  • 5
    @BrianPeterson it would be something like that: `find . -iname "*.sql" -print0 | xargs -0 grep "slow"` – MediaVince Aug 18 '16 at 08:41
  • `grep -r` offers much the same functionality. The OP didn't seem to actually want to traverse subdirectories. – tripleee Nov 24 '20 at 18:20
  • 4
    `-exec` on modern `find` supports shortcutting just like `xargs`, and will actually be faster (because you cut out `xargs`). Use `-exec ... {} +` instead of `-exec ... {} \;` – tripleee Nov 24 '20 at 18:21
20

To search in all sub-directories, but only in specific file types, use grep with --include.

For example, searching recursively in current directory, for text in *.yml and *.yaml :

grep "text to search" -r . --include=*.{yml,yaml}
Noam Manos
  • 15,216
  • 3
  • 86
  • 85
1

If you want to do multiple commands, you could use:

for I in `ls *.sql`
do
    grep "foo" $I >> foo.log
    grep "bar" $I >> bar.log
done
bryan
  • 798
  • 7
  • 18
  • Generally, probably try http://shellcheck.net/ on the code you post. [Don't use `ls` to expand a wildcard](https://mywiki.wooledge.org/ParsingLs) (the shell already does it for you!) and [quote your shell variables](https://stackoverflow.com/questions/10067266/when-to-wrap-quotes-around-a-shell-variable). – tripleee Nov 24 '20 at 18:19
  • You probably want to use Awk instead for this, especially if the input file is big. `awk '/foo/ { print >"foo.log" } /bar/ { print >"bar.log" }' "$I"` searches for both patterns in a single pass over the input file. (Try `grep -E 'foo|bar' "$I"`if you want to search for multiple patterns, but don't require multiple output files.) – tripleee Nov 24 '20 at 18:25