confused about what must be escaped for sed

Question

I want to replace specific strings in php files automatically using sed. Some work, and some do not. I already investigated this is not an issue with the replacement string but with the string that is to be replaced. I already tried to escape [ and ] with no success. It seems to be the whitespace within the () - not whitespaces in general. The first whitespaces (around the = ) do not have any problems. Please can someone point me to the problem:

sed -e "1,\$s/$adm = substr($path . rawurlencode($upload['name']) , 16);/$adm = rawurlencode($upload['name']); # fix 23/g" -i administration/identify.php

I already tried to shorten the string which should be replaced and the result was if I cut it directly behind $path it works, with the following whitespace it does not. Escaping whitespace has no effect...

you don't need `1,$` as address range as that is the default... use single quotes instead of double quotes around the sed expression unless `$adm`, `$path`, etc are all bash variables? — Sundeep, May 01 '20 at 12:31
you'd need to escape `[` character, and use `\x27` where single quotes are needed, for example, `\x27name\x27` — Sundeep, May 01 '20 at 12:33
Try `sed -i 's/\$adm = substr(\$path \. rawurlencode(\$upload\['"'"'name'"'"']) , 16);/$adm = rawurlencode($upload['"'"'name'"'"']); # fix 23/g' administration/identify.php` — Wiktor Stribiżew, May 01 '20 at 12:34
nothing of this are bash variables - all has to be interpreted as plain text — DF8OE, May 01 '20 at 12:36
this works: sed -e "s/$adm = substr($path/..... and this does not work: sed -e "s/$adm = substr($path /.... this does not work, too: sed -e "s/$adm = substr($path\ /.... and replacing whitepsace after $path with /s does not work, too. It seems to be a problem with the whitespace following $path (maybe not the only issue) — DF8OE, May 01 '20 at 12:44
If my answer lacks something please consider updating the question. — Wiktor Stribiżew, Nov 28 '21 at 19:24

KamilCuk · Answer 1 · 2021-03-22T20:20:28.167

what must be escaped for sed

The following characters have special meaning in sed and have to be escaped with \ for the regex to be taken literally:

\
[
the character used in separating s command parts, ie. / here
.
*
& only replacement string

Newline character is handled specially as the end of the string, but can be replaced for \n.

So first escape all special characters in input and then pass it to sed:

rgx="$adm = substr($path . rawurlencode($upload['name']) ,  16);"
rgx_escaped=$(sed 's/[\\\[\.\*\/&]/\\&/g' <<<"$rgx")
sed "s/$rgx_escaped/ etc."

See Escape a string for a sed replace pattern for a generic escaping solution.

score 1 · Answer 2 · answered May 01 '20 at 12:43

You may use

sed -i 's/\$adm = substr(\$path \. rawurlencode(\$upload\['"'"'name'"'"']) , 16);/$adm = rawurlencode($upload['"'"'name'"'"']); # fix 23/g' administration/identify.php

Note:

the sed command is basically wrapped in single quotes, the variable expansion won't occur inside single quotes
In the POSIX BRE syntax, ( matches a literal (, you do not need to escape ) either, but you need to escape [ and . that must match themselves
The single quotes require additional quoting with concatenation.

confused about what must be escaped for sed

2 Answers2