449

I have a file that contains directory names:

my_list.txt :

/tmp
/var/tmp

I'd like to check in Bash before I'll add a directory name if that name already exists in the file.

codeforester
  • 39,467
  • 16
  • 112
  • 140
Toren
  • 6,648
  • 12
  • 41
  • 62
  • To find all of the strings inside a file, you can run grep in FOR loop: https://unix.stackexchange.com/a/462445/43233 – Noam Manos Aug 14 '18 at 06:59
  • [find . -type f -print0 | xargs -0 grep -l "some string"](https://superuser.com/questions/614526/finding-files-which-contain-a-certain-string-using-find-1-and-grep-1) – Nick Dong Dec 08 '22 at 07:05
  • [grep -r --include '*.java' teststring /home/user1](https://superuser.com/questions/215896/search-for-a-text-pattern-in-linux#215904) – Nick Dong Dec 08 '22 at 07:15

16 Answers16

837
grep -Fxq "$FILENAME" my_list.txt

The exit status is 0 (true) if the name was found, 1 (false) if not, so:

if grep -Fxq "$FILENAME" my_list.txt
then
    # code if found
else
    # code if not found
fi

Explanation

Here are the relevant sections of the man page for grep:

grep [options] PATTERN [FILE...]

-F, --fixed-strings

        Interpret PATTERN as a list of fixed strings, separated by newlines, any of which is to be matched.

-x, --line-regexp

        Select only those matches that exactly match the whole line.

-q, --quiet, --silent

        Quiet; do not write anything to standard output. Exit immediately with zero status if any match is found, even if an error was detected. Also see the -s or --no-messages option.

Error handling

As rightfully pointed out in the comments, the above approach silently treats error cases as if the string was found. If you want to handle errors in a different way, you'll have to omit the -q option, and detect errors based on the exit status:

Normally, the exit status is 0 if selected lines are found and 1 otherwise. But the exit status is 2 if an error occurred, unless the -q or --quiet or --silent option is used and a selected line is found. Note, however, that POSIX only mandates, for programs such as grep, cmp, and diff, that the exit status in case of error be greater than 1; it is therefore advisable, for the sake of portability, to use logic that tests for this general condition instead of strict equality with 2.

To suppress the normal output from grep, you can redirect it to /dev/null. Note that standard error remains undirected, so any error messages that grep might print will end up on the console as you'd probably want.

To handle the three cases, we can use a case statement:

case `grep -Fx "$FILENAME" "$LIST" >/dev/null; echo $?` in
  0)
    # code if found
    ;;
  1)
    # code if not found
    ;;
  *)
    # code if an error occurred
    ;;
esac
Community
  • 1
  • 1
Thomas
  • 174,939
  • 50
  • 355
  • 478
  • 2
    If I execute this command from bash script how to catch 0 or 1 into a variable ? – Toren Jan 20 '11 at 16:06
  • 7
    @Toren Most recent exit status can be accessed using `$?`. you can also use the grep command alongside the `if` statement (as shown in updated answer). – Shawn Chin Jan 20 '11 at 16:08
  • 5
    You can use `grep -Fqx "$FILENAME"` and you don't have to worry about regex characters in the variable contents and you won't have to use them in the search string. – Dennis Williamson Jan 20 '11 at 16:49
  • Ah, good one! Didn't know about `-x` (and `-F` by itself was obviously not enough). Edited. – Thomas Jan 20 '11 at 16:51
  • Can you please provide some example of using grep -Fqx ? I didn't get it what it good for . 10x – Toren Jan 23 '11 at 09:42
  • @Toren: It's already there in my answer. Or isn't that what you meant? – Thomas Jan 24 '11 at 08:28
  • @Thomas :I understand why use -q , but can you add some more explanation why to use -Fx . I do not understand it from your comment or from grep manual. Thank you in advance. – Toren Jan 24 '11 at 09:21
  • 1
    @Toren: Without `-F`, your filename would be interpreted as a regular expression, not as a normal string. Thus, if it contains strange characters (like `.`), these will be interpreted by `grep`'s regex engine, and not matched literally. The `-x` flag ensures that the string matches an entire line, not just a part of it: so if you search for `foo`, only lines containing exactly `foo` are matched, not `foobar`. (There was an error in my examples; it still used some regex syntax. Fixed now.) – Thomas Jan 24 '11 at 17:04
  • To be good when file not exists and avoid getting errors in stderr use `if grep -OPTIONS "" file.txt &> /dev/null; then` – wonder.mice Nov 01 '13 at 02:06
  • You'd probably want to know that that happened (e.g. if you ran the script from the wrong directory), instead of silently giving the wrong answer. – Thomas Nov 01 '13 at 19:42
  • This even works in the limited `/bin/sh` offered by VMware's ESXi 5.5 :) – Aaron R. Feb 25 '15 at 18:50
  • Probably `grep` isn't part of `/bin/sh` but a separate program. All flags in use here are [part of POSIX](http://pubs.opengroup.org/onlinepubs/009604499/utilities/grep.html) so you can expect them to be available either way. – Thomas Mar 02 '15 at 15:54
  • How can you do this with a variable? If i store the result of grep in a variable, I can't get it to work. – B T Nov 17 '15 at 09:25
  • @BT: `grep ...`, then assign like `grep_status=$?` and test it with `if [[ $grep_status -eq 0 ]]`. – Thomas Nov 17 '15 at 14:16
  • I have to swap found and not found branches for this to work – andrej Sep 10 '16 at 19:01
  • @andrej: That's really weird, and suggests that your `grep` is not POSIX compliant (see [here](http://pubs.opengroup.org/onlinepubs/009604499/utilities/grep.html)). What OS/distro/grep are you using? What does `grep -Fxq foo <(echo foo); echo $?` output (I get 0)? How about `grep -Fxq foo <(echo bar); echo $?` (I get 1)? – Thomas Sep 12 '16 at 09:23
  • @thomas 0 and 1 as for you. I have to doublecheck my previous check. – andrej Sep 12 '16 at 20:26
  • 4
    A couple of notes for folks looking at this answer: 1) In bash, 0 is always true and anything else is always false 2) Only use the -x flag if you want the entire line to match exactly. If you just want to find if your string exists in the file at all, leave that off. If you want to find if your string exists exactly but without matching an entire line necessarily (i.e., as a whole word), use -w. – Schmick Mar 09 '17 at 21:52
  • 1
    I do not gethe the `-q / --silent` is it needed? Its falsely says "all good" to bash even if a error occurs. If I got that correctly. Seems a flawed concept for this case. – redanimalwar Sep 07 '19 at 14:43
  • I am still confused why not just do it without `-q`. I do not want errors to be ignored, I also do not want a complicated case statement. What is so bad in having the grep output in a script? Seems stupid to me that it just ignores errors. From the man: `Portable shell scripts should avoid both -q and -s and should redirect standard and error output to /dev/null instead. (-s is specified by POSIX.)` But here you and the other guys give people advice to do exactly that! – redanimalwar Sep 07 '19 at 18:44
  • @redanimalwar In my edited version at the bottom of this post, I'm doing exactly that (`do it without -q` and `avoid both -q and -s and should redirect standard and error output to /dev/null instead`). The problem is fundamental: there are three possible outcomes (found, not found, error) so how can you possibly expect to handle it with a single `if` statement? Instead of the `complicated case statement` you can use two `if`s if you prefer, but I think the `case` is easier to read. – Thomas Sep 09 '19 at 08:09
109

Regarding the following solution:

grep -Fxq "$FILENAME" my_list.txt

In case you are wondering (as I did) what -Fxq means in plain English:

  • F: Affects how PATTERN is interpreted (fixed string instead of a regex)
  • x: Match whole line
  • q: Shhhhh... minimal printing

From the man file:

-F, --fixed-strings
    Interpret  PATTERN  as  a  list of fixed strings, separated by newlines, any of which is to be matched.
    (-F is specified by POSIX.)
-x, --line-regexp
    Select only those matches that exactly match the whole line.  (-x is specified by POSIX.)
-q, --quiet, --silent
    Quiet; do not write anything to standard output.  Exit immediately with zero status  if  any  match  is
          found,  even  if  an error was detected.  Also see the -s or --no-messages option.  (-q is specified by
          POSIX.)
Eje
  • 354
  • 4
  • 8
Kuf
  • 17,318
  • 6
  • 67
  • 91
  • 5
    -F does not affect the file processing, it affects how PATTERN is interpreted. Typically, PATTERN is interpreted as a regex, but with -F it will be interpreted as a fixed string. – Adam S Mar 12 '13 at 19:57
51

Three methods in my mind:

1) Short test for a name in a path (I'm not sure this might be your case)

ls -a "path" | grep "name"


2) Short test for a string in a file

grep -R "string" "filepath"


3) Longer bash script using regex:

#!/bin/bash

declare file="content.txt"
declare regex="\s+string\s+"

declare file_content=$( cat "${file}" )
if [[ " $file_content " =~ $regex ]] # please note the space before and after the file content
    then
        echo "found"
    else
        echo "not found"
fi

exit

This should be quicker if you have to test multiple string on a file content using a loop for example changing the regex at any cicle.

Luca Borrione
  • 16,324
  • 8
  • 52
  • 66
32

Easiest and simplest way would be:

isInFile=$(cat file.txt | grep -c "string")


if [ $isInFile -eq 0 ]; then
   #string not contained in file
else
   #string is in file at least once
fi

grep -c will return the count of how many times the string occurs in the file.

Christian737
  • 421
  • 4
  • 4
25

Simpler way:

if grep "$filename" my_list.txt > /dev/null
then
   ... found
else
   ... not found
fi

Tip: send to /dev/null if you want command's exit status, but not outputs.

imwilsonxu
  • 2,942
  • 24
  • 25
10

Here's a fast way to search and evaluate a string or partial string:

if grep -R "my-search-string" /my/file.ext
then
    # string exists
else
    # string not found
fi

You can also test first, if the command returns any results by running only:

grep -R "my-search-string" /my/file.ext
GTodorov
  • 1,993
  • 21
  • 24
7
grep -E "(string)" /path/to/file || echo "no match found"

-E option makes grep use regular expressions

David Okwii
  • 7,352
  • 2
  • 36
  • 29
6

If I understood your question correctly, this should do what you need.

  1. you can specifiy the directory you would like to add through $check variable
  2. if the directory is already in the list, the output is "dir already listed"
  3. if the directory is not yet in the list, it is appended to my_list.txt

In one line: check="/tmp/newdirectory"; [[ -n $(grep "^$check\$" my_list.txt) ]] && echo "dir already listed" || echo "$check" >> my_list.txt

lecodesportif
  • 10,737
  • 9
  • 38
  • 58
  • You don't need to test the output of grep, you can just use `grep -q` and call grep directly from `if` as Thomas does in his answer. In addition, the question didn't include checking whether the directory exists before adding it to the list (it could be a list of deleted directories, after all). – sorpigal Jan 20 '11 at 17:26
  • I removed the example script, it didn't add anything to the answer given by Thomas. – lecodesportif Jan 25 '11 at 13:38
5

The @Thomas's solution didn't work for me for some reason but I had longer string with special characters and whitespaces so I just changed the parameters like this:

if grep -Fxq 'string you want to find' "/path/to/file"; then
    echo "Found"
else
    echo "Not found"
fi

Hope it helps someone

B8ightY
  • 469
  • 5
  • 5
4

If you just want to check the existence of one line, you do not need to create a file. E.g.,

if grep -xq "LINE_TO_BE_MATCHED" FILE_TO_LOOK_IN ; then
  # code for if it exists
else
  # code for if it does not exist
fi  
gordon
  • 51
  • 6
4

My version using fgrep

  FOUND=`fgrep -c "FOUND" $VALIDATION_FILE`
  if [ $FOUND -eq 0 ]; then
    echo "Not able to find"
  else
    echo "able to find"     
  fi  
Rudy
  • 7,008
  • 12
  • 50
  • 85
2

I was looking for a way to do this in the terminal and filter lines in the normal "grep behaviour". Have your strings in a file strings.txt:

string1
string2
...

Then you can build a regular expression like (string1|string2|...) and use it for filtering:

cmd1 | grep -P "($(cat strings.txt | tr '\n' '|' | head -c -1))" | cmd2

Edit: Above only works if you don't use any regex characters, if escaping is required, it could be done like:

cat strings.txt | python3 -c "import re, sys; [sys.stdout.write(re.escape(line[:-1]) + '\n') for line in sys.stdin]" | ...
jns
  • 1,250
  • 1
  • 13
  • 29
1

A grep-less solution, works for me:

MY_LIST=$( cat /path/to/my_list.txt )



if [[ "${MY_LIST}" == *"${NEW_DIRECTORY_NAME}"* ]]; then
  echo "It's there!"
else
echo "its not there"
fi

based on: https://stackoverflow.com/a/229606/3306354

AndrewD
  • 4,924
  • 3
  • 30
  • 32
  • 1
    Uses too much memory in case the file is large. `grep -q` described in the [accepted answer](https://stackoverflow.com/a/4749368/6862601) is the most efficient approach. – codeforester May 09 '18 at 01:29
1
grep -Fxq "String to be found" | ls -a
  • grep will helps you to check content
  • ls will list all the Files
Shinoy Shaji
  • 397
  • 10
  • 27
1

Slightly similar to other answers but does not fork cat and entries can contain spaces

contains() {
    [[ " ${list[@]} " =~ " ${1} " ]] && echo 'contains' || echo 'does not contain'
}

IFS=$'\r\n' list=($(<my_list.txt))

so, for a my_list.txt like

/tmp
/var/tmp
/Users/usr/dir with spaces

these tests

contains '/tmp'
contains '/bin'
contains '/var/tmp'
contains '/Users/usr/dir with spaces'
contains 'dir with spaces'

return

exists
does not exist
exists
exists
does not exist
Diego Torres Milano
  • 65,697
  • 9
  • 111
  • 134
-1
if grep -q "$Filename$" my_list.txt
   then
     echo "exist"
else 
     echo "not exist"
fi
jch
  • 5,382
  • 22
  • 41
Triangle
  • 1,477
  • 3
  • 22
  • 36