Replace the string content in sed with special chars

Question

I have the code like this:

sed "s/TEST_CASES_R/$testCaseLocations/g" template >> newfile

where $testCaseLocations has ,tests/test/,tests/test/2. So this line is failing like:

bad flag in substitute command

How can I solve this?

Answer 1

Ah, sed code injection. What sed sees is

sed "s/TEST_CASES_R/,tests/test/,tests/test/2/g" template >> newfile

...which is nonsensical. The root problem is that sed cannot differentiate between the things you want it to see as data -- the contents of $testCaseLocations -- and instructions.

The best solution in my opinion is to use awk:

awk -v replacement="$testCaseLocations" '{ gsub(/TEST_CASES_R/, replacement); print }' template >> newfile

because this neatly sidesteps code injection problems by not treating testCaseLocations as code. You could, in this particular case, also use a different delimiter for sed, such as

 sed "s@TEST_CASES_R@$testCaseLocations@g" template >> newfile

but then you'd run into trouble if $testCaseLocations contains a @, or if it contains a character that has meaning for sed in the context where it appears, such as \ or &.

Answer 2

Just use another separator for sed, otherwise it sees so many slashes around: sed 's#hello#bye#g' is fine.

In your case:

sed "s#TEST_CASES_R#$testCaseLocations#g" template >> newfile

See another test:

$ var="/hello"
$ echo "test" | sed "s/test/$var/g"
sed: -e expression #1, char 9: unknown option to `s'
$ echo "test" | sed "s#test#$var#g"
/hello