1

I've got two variables VAR1 and VAR2 that contain strings. What I want to do is go through a list of files that have a .txt extension and change all occurences of VAR1 to VAR2. So far, it looks like this:

for i in `find . -name "*.txt"`
do
 echo $i
 sed -i -E "s|\$VAR1|\$VAR2|g" $i
done

I think everything except the sed line is working well. I think it's a syntax issue, but I haven't been able to figure out what it is. Any help would be appreciated Thanks

Many Questions
  • 125
  • 1
  • 10
  • Note that on Mac OS X (and BSD), `sed` will backup your files with the `-E` suffix (so `xyz.txt` will be backed up to `xyz.txt-E`). GNU `sed` will handle things differently (the backup suffix, if it exists, must be attached to the `-i` option). When you're not running into that issue, the `-E` option in Mac OS X (BSD) `sed` means 'use extended regular expressions' instead of '`sed`-style basic regular expressions'. The normal equivalent option for GNU `sed` is `-r`. – Jonathan Leffler Jan 25 '16 at 02:57
  • 1
    You seem to want to replace `$VAR1` with `$VAR2` only when there is a dollar sign in front. Is this correct? Please [edit] your question to clarify. Several of the answers here assume you want to interpolate the values of the shell variables `VAR1` and `VAR2` into the `sed` script instead. – tripleee Jan 25 '16 at 08:19
  • 1
    Please [edit your question](http://stackoverflow.com/posts/34983968/edit) to include a [minimal example](http://stackoverflow.com/help/mcve) that demonstrates what you're getting at. For example, a list of files, both before and after running the script. Including examples will provide clarity that may not be available from the description or code you've provided. – ghoti Jan 25 '16 at 08:24

4 Answers4

2

You shouldn't need to escape your $ variable. Also make sure to use the lower case -e and quote the filename in case it has spaces:

sed -ri -e "s|$VAR1|$VAR2|g" "$i"
Martin Konecny
  • 57,827
  • 19
  • 139
  • 159
1

Since sed's "find-and-replace" functionality is oriented to regular expressions rather than literal strings, you might wish to consider an alternative to sed, e.g. using awk as follows:

 awk -v from="$VAR1" -v to="$VAR2" '
   function replace(a,b,s,  n) {
     n=index(s,a);
     if (n==0) {return s}
     return substr(s,1,n-1) b replace(a,b, substr(s,n+length(a)));
   }
   {print replace(from, to, $0)} '

The above can easily be combined with the find ... | while read f ; do .... done pattern mentioned elsewhere on this page.

GNU awk supports the equivalent of sed's '-i' option, but it's probably better simply to direct the output of awk to a temporary file, and then mv it into place.

peak
  • 105,803
  • 17
  • 152
  • 177
1

You managed to quote the dollar sign from the shell (which would not have been necessary if you had used single quotes instead of double) but this does not change the fact that dollar signs also have a meaning in regular expressions. Double the backslashes to escape from both the shell and sed, or use single quotes so the backslashes get through to sed. Alternatively, use a notation which does not require backslashes.

sed -i -E 's|[$]VAR1|$VAR2|g' "$i"

Incidentally, your loop has a number of problems. Your for loop will not work correctly if there are file names with whitespace in them, and you need to quote the arguments inside the loop. To completely cope with file names with special characters in them, you want to use find -exec instead.

find . -name "*.txt" -exec sed -i -E 's|[$]VAR1|$VAR2|g' {} \;

If your find supports \+ instead of \;, by all means use that.

Community
  • 1
  • 1
tripleee
  • 175,061
  • 34
  • 275
  • 318
  • Great answer. I would like to also suggest ( as a best practice ) to write VAR1 and VAR2 in lower case, since by convention, environment variables (PATH, EDITOR, SHELL, ...) and internal shell variables (BASH_VERSION, RANDOM, ...) are fully capitalized. All other variable names should be lowercase. Since variable names are case-sensitive, this convention avoids accidentally overriding environmental and internal variables. – Rany Albeg Wein Jan 25 '16 at 23:03
0

(1) Using the idiom for i in $(find ....) ; do ...; done will often work as intended, but it is not robust. Significantly better is the pattern:

find ... | while read i ; do ... ; done

(2) If $VAR1 and/or $VAR2 contain characters that have special significance in regular expressions, then some care will be required. For example, parentheses ("(" and ")") have special significance, and so if VAR1 contains these, using the -r option (or on a Mac, the -E option) is probably asking for trouble.

(3) Chances are that sed -i -e "s|$VAR1|$VAR2|g" will do the trick if VAR1 does not contain any of the eight characters: ^$*[]\|. and if VAR2 does not contain "|", "\" or "&".

(4) If you want to prepare your strings ($VAR1 and $VAR2) programatically for use with sed, then see this SO page; it shows how to munge the strings -- using sed of course!

Community
  • 1
  • 1
peak
  • 105,803
  • 17
  • 152
  • 177
  • Never do this: for x in $(command) or `command` or $var. for-in is used for iterating arguments, not (output) strings. Instead, use a glob (eg. *.txt), arrays (eg. "${names[@]}") or a while-read loop (eg. while read -r line). See http://mywiki.wooledge.org/BashPitfalls#pf1 and http://mywiki.wooledge.org/DontReadLinesWithFor – Rany Albeg Wein Jan 25 '16 at 14:28
  • @Rany Albeg Wein - Your comment would be better placed with one of the posts that suggest using "for", wouldn't it? Here, your comment mainly reiterates (1). – peak Jan 25 '16 at 17:22
  • Just giving a side note so that people won't run dangerous stuff as you suggested `"will often work as intended, but it is not robust."` The provided links show some great examples of the issue. – Rany Albeg Wein Jan 25 '16 at 17:28