How to check if sed has changed a file

Question

I am trying to find a clever way to figure out if the file passed to sed has been altered successfully or not.

Basically, I want to know if the file has been changed or not without having to look at the file modification date.

The reason why I need this is because I need to do some extra stuff if sed has successfully replaced a pattern.

I currently have:

    grep -q $pattern $filename
    if [ $? -eq 0 ]
    then
        sed -i s:$pattern:$new_pattern: $filename
                # DO SOME OTHER STUFF HERE
    else
        # DO SOME OTHER STUFF HERE
    fi

The above code is a bit expensive and I would love to be able to use some hacks here.

Answer 1

A bit late to the party but for the benefit of others, I found the 'w' flag to be exactly what I was looking for.

sed -i "s/$pattern/$new_pattern/w changelog.txt" "$filename"
if [ -s changelog.txt ]; then
    # CHANGES MADE, DO SOME STUFF HERE
else
    # NO CHANGES MADE, DO SOME OTHER STUFF HERE
fi

changelog.txt will contain each change (ie the changed text) on it's own line. If there were no changes, changelog.txt will be zero bytes.

A really helpful sed resource (and where I found this info) is http://www.grymoire.com/Unix/Sed.html.

Answer 2

I believe you may find these GNU sed extensions useful

t label

If a s/// has done a successful substitution since the last input line
was read and since the last t or T command, then branch to label; if
label is omitted, branch to end of script.

and

q [exit-code]

Immediately quit the sed script without processing any more input, except 
that if auto-print is not disabled the current pattern space will be printed. 
The exit code argument is a GNU extension.

It seems like exactly what are you looking for.

Answer 3

This might work for you (GNU sed):

sed -i.bak '/'"$old_pattern"'/{s//'"$new_pattern"'/;h};${x;/./{x;q1};x}' file || echo changed

Explanation:

• /'"$old_pattern"'/{s//'"$new_pattern"'/;h} if the pattern space (PS) contains the old pattern, replace it by the new pattern and copy the PS to the hold space (HS).
• ${x;/./{x;q1};x} on encountering the last line, swap to the HS and test it for the presence of any string. If a string is found in the HS (i.e. a substitution has taken place) swap back to the original PS and exit using the exit code of 1, otherwise swap back to the original PS and exit with the exit code of 0 (the default).

Answer 4

You can diff the original file with the sed output to see if it changed:

sed -i.bak s:$pattern:$new_pattern: "$filename"
if ! diff "$filename" "$filename.bak" &> /dev/null; then
  echo "changed"
else
  echo "not changed"
fi
rm "$filename.bak"

Answer 5

You could use awk instead:

awk '$0 ~ p { gsub(p, r); t=1} 1 END{ exit (!t) }' p="$pattern" r="$repl"

I'm ignoring the -i feature: you can use the shell do do redirections as necessary.

Sigh. Many comments below asking for basic tutorial on the shell. You can use the above command as follows:

if awk '$0 ~ p { gsub(p, r); t=1} 1 END{ exit (!t) }' \
        p="$pattern" r="$repl" "$filename" > "${filename}.new"; then
    cat "${filename}.new" > "${filename}"
    # DO SOME OTHER STUFF HERE
else
    # DO SOME OTHER STUFF HERE
fi

It is not clear to me if "DO SOME OTHER STUFF HERE" is the same in each case. Any similar code in the two blocks should be refactored accordingly.

Answer 6

In macos I just do it as follows:

changes=""
changes+=$(sed -i '' "s/$to_replace/$replacement/g w /dev/stdout" "$f")
if [ "$changes" != "" ]; then
  echo "CHANGED!"
fi

I checked, and this is faster than md5, cksum and sha comparisons

Answer 7

I know it is a old question and using awk instead of sed is perhaps the best idea, but if one wants to stick with sed, an idea is to use the -w flag. The file argument to the w flag only contains the lines with a match. So, we only need to check that it is not empty.

Answer 8

perl -sple '$replaced++ if s/$from/$to/g;
                END{if($replaced != 0){ print "[Info]: $replaced replacement done in $ARGV(from/to)($from/$to)"}
                else {print "[Warning]: 0 replacement done in $ARGV(from/to)($from/$to)"}}' -- -from="FROM_STRING" -to="$DESIRED_STRING" </file/name>

Example: The command will produce the following output, stating the number of changes made/file.

perl -sple '$replaced++ if s/$from/$to/g;
END{if($replaced != 0){ print "[Info]: $replaced replacement done in $ARGV(from/to)($from/$to)"}
else {print "[Warning]: 0 replacement done in $ARGV(from/to)($from/$to)"}}' -- -from="timeout" -to="TIMEOUT" *
[Info]: 5 replacement done in main.yml(from/to)(timeout/TIMEOUT)
[Info]: 1 replacement done in task/main.yml(from/to)(timeout/TIMEOUT)
[Info]: 4 replacement done in defaults/main.yml(from/to)(timeout/TIMEOUT)
[Warning]: 0 replacement done in vars/main.yml(from/to)(timeout/TIMEOUT) 

Note: I have removed -i from the above command , so it will not update the files for the people who are just trying out the command. If you want to enable in-place replacements in the file add -i after perl in above command.

Answer 9

check if sed has changed MANY files

• recursive replace of all files in one directory
• produce a list of all modified files

workaround with two stages: match + replace

g='hello.*world'
s='s/hello.*world/bye world/g;'
d='./' # directory of input files
o='modified-files.txt'

grep -r -l -Z -E "$g" "$d" | tee "$o" | xargs -0 sed -i "$s"

the file paths in $o are zero-delimited

Answer 10

$ echo hi > abc.txt
$ sed "s/hi/bye/g; t; q1;" -i abc.txt && (echo "Changed") || (echo "Failed")
Changed
$ sed "s/hi/bye/g; t; q1;" -i abc.txt && (echo "Changed") || (echo "Failed")
Failed

https://askubuntu.com/questions/1036912/how-do-i-get-the-exit-status-when-using-the-sed-command/1036918#1036918

Answer 11

The problem with s/foo/bar/;t;q8 is that is quits after the first line. To set a proper exit code for multi-line or multi-substitution situations you can use the following:

sed -i "s/foo/bar/w /dev/stdout" dir/file.txt | grep -q .
echo $?

which uses -i to change the file in-place, the w command to send the successful substitutions to /dev/stdout, then we read that with grep . to see if output is empty. This returns an exit code of 1 when no match is found, and 0 when substitutions have been made. Use this in conjunction with set -e for example.

Answer 12

Don't use sed to tell if it has changed a file; instead, use grep to tell if it is going to change a file, then use sed to actually change the file. Notice the single line of sed usage at the very end of the Bash function below:

# Usage: `gs_replace_str "regex_search_pattern" "replacement_string" "file_path"`
gs_replace_str() {
    REGEX_SEARCH="$1"
    REPLACEMENT_STR="$2"
    FILENAME="$3"

    num_lines_matched=$(grep -c -E "$REGEX_SEARCH" "$FILENAME")
    # Count number of matches, NOT lines (`grep -c` counts lines), 
    # in case there are multiple matches per line; see: 
    # https://superuser.com/questions/339522/counting-total-number-of-matches-with-grep-instead-of-just-how-many-lines-match/339523#339523
    num_matches=$(grep -o -E "$REGEX_SEARCH" "$FILENAME" | wc -l)

    # If num_matches > 0
    if [ "$num_matches" -gt 0 ]; then
        echo -e "\n${num_matches} matches found on ${num_lines_matched} lines in file"\
                "\"${FILENAME}\":"
        # Now show these exact matches with their corresponding line 'n'umbers in the file
        grep -n --color=always -E "$REGEX_SEARCH" "$FILENAME"
        # Now actually DO the string replacing on the files 'i'n place using the `sed` 
        # 's'tream 'ed'itor!
        sed -i "s|${REGEX_SEARCH}|${REPLACEMENT_STR}|g" "$FILENAME"
    fi
}

Place that in your ~/.bashrc file, for instance. Close and reopen your terminal and then use it.

Usage:

gs_replace_str "regex_search_pattern" "replacement_string" "file_path"

Example: replace do with bo so that "doing" becomes "boing" (I know, we should be fixing spelling errors not creating them :) ):

$ gs_replace_str "do" "bo" test_folder/test2.txt 

9 matches found on 6 lines in file "test_folder/test2.txt":
1:hey how are you doing today
2:hey how are you doing today
3:hey how are you doing today
4:hey how are you doing today  hey how are you doing today  hey how are you doing today  hey how are you doing today
5:hey how are you doing today
6:hey how are you doing today?
$SHLVL:3 

Screenshot of the output:

References:

1. https://superuser.com/questions/339522/counting-total-number-of-matches-with-grep-instead-of-just-how-many-lines-match/339523#339523
2. https://unix.stackexchange.com/questions/112023/how-can-i-replace-a-string-in-a-files/580328#580328