Substitute a text with long string (which varies every time) that contains special characters

Question

Find and replace a text in a file. Find word is same all the time and replacing string varies all the time

I am trying a simple sed script (test.sh) to achieve this which uses a parameter(a long string with special characters):

sed -i "s/#####REPLACE#####/$1/g" file.txt

The result should replace #####REPLACE##### with the parameter that I pass to the script test.sh.

The parameter varies all the time and contains a lot of special characters (something like this):

"abc"/def:-/ghi$jklmno-pqrblah..blah..very..long..string

The command I am trying is:

./test.sh "abc"/def:-/ghi$jklmno-pqrblah..blah..very..long..string

It gives out the below message:

sed: -e expression #1, char 25: unknown option to `s'

I have many files (e.g. file1.txt, file2.txt, file3.txt) with the same common text #####REPLACE##### to be replaced by the same parameter "abc"/def:-/ghi$jklmno-pqrblah..blah..very..long..string.

Any suggestions?

Answer 1

One option would be to use awk instead:

awk -v replace="$1" '{ gsub(/#####REPLACE#####/, replace); print }' file

This sets an awk variable replace equal to the first argument passed to the function $1, then performs a global substitution on each line of file.

To overwrite the file "in-place", just use awk ... file > tmp && mv tmp file.

The advantage of this approach over using sed is that it doesn't matter what characters occur in the replacement string*; they will never cause a syntax error.

_{^{*As pointed out in the comments, there is one character that will break things: &}}

---

As you're only really interested in performing a string replacement (not a regex substitution), another option would be to use something like this:

awk -v replace="$1" '{
    out = ""
    while(match($0, /#####REPLACE#####/)) {
        out = out substr($0, 0, RSTART - 1) replace
        $0 = substr($0, RSTART + RLENGTH)
    }
print out $0}' file

This builds up a replacement string out using a combination of the input record $0 and the variable replace.

Answer 2

Problem is that you're using / as regex delimiter in sed and same character is part of replacement string also. Use an alternate regex delimiter in sed:

sed -i "s~#####REPLACE#####~$1~g" file.txt

This is assuming ~ (tilde) will not be in the replacement text. If that is the case then you can even use a control character such as as ^A ctrlVA (pressed together) as regex delimiter.

sed -i "s^A#####REPLACE#####^A$1^Ag" file.txt

PS: If you want to pass & as literal & in $1 then escape it as \&