Escaping a variable with special characters within sed - comment and uncomment an arbitrary line of source code

Question

I need to comment out a line in a crontab file through a script, so it contains directories, spaces and symbols. This specific line is stored in a variable and I am starting to get mixed up on how to escape the variable. Since the line changes on a regular basis I dont want any escaping in there. I don't want to simply add # in front of it, since I also need to switch it around and replace the line again with the original without the #.

So the goal is to replace $line with #$line (comment) with the possibility to do it the other way around (uncomment).

So I have a variable:

line="* * * hello/this/line & /still/this/line"

This is a line that occurs in a file, file.txt. Wich needs to get comment out.

First try:

sed -i "s/^${line}/#${line}/" file.txt

Second try:

sed -i 's|'${line}'|'"#${line}"'|g' file.txt

As for your own tries: The 1st try fails, because the value of `$line` contains characters that need escaping for use with `s` (the crux of the matter; in this _particular_ case, using `|` as the delimiter would have worked, but that's not a general solution). The 2nd try makes matters worse, because the _unquoted_ use of `$line` now causes the shell to perform word-splitting and globbing on the value. — mklement0, Jan 21 '17 at 03:11

choroba · Answer 1 · 2017-01-21T13:51:48.787

1

Perl has ways to do the quoting/escaping for you:

line=$line perl -i~ -pe '$regex = quotemeta $ENV{line}; s/^$regex/#$ENV{line}/' -- input.txt

edited Jan 21 '17 at 13:51

answered Jan 20 '17 at 08:30

choroba

231,213
25
204
289

This adds a # for every line in file.txt – Swifting Jan 20 '17 at 12:21
Thnx, that worked, but how would we adjust this syntax do to it the other way around? – Swifting Jan 20 '17 at 12:28
@Swifting: What do you mean by "other way around"? – choroba Jan 20 '17 at 12:31
Changing #$ENV{line} in $ENV{line} again. – Swifting Jan 20 '17 at 12:32
The line variable needed an export first in order for your line to work. But I was wondering how to change the syntax to revert the line back to the original, without the # in front of it. – Swifting Jan 20 '17 at 13:57
Might be something like 's/#ENV{line}/ENV{line}/g'? – Swifting Jan 20 '17 at 14:00

score 1 · Accepted Answer · edited May 23 '17 at 11:52

choroba's helpful answer shows an effective solution using perl.

`sed` solution

If you want to use sed, you must use a separate sed command just to escape the $line variable value, because sed has no built-in way to escape strings for use as literals in a regex context:

lineEscaped=$(sed 's/[^^]/[&]/g; s/\^/\\^/g' <<<"$line") # escape $line for use in regex
sed -i "s/^$lineEscaped\$/#&/" file.txt # Note the \$ to escape the end-of-line anchor $

^{With BSD/macOS sed, use -i '' instead of just -i for in-place updating without backup.}

And the reverse (un-commenting):

sed -i "s/^#\($lineEscaped\)\$/\1/" file.txt

See this answer of mine for an explanation of the sed command used for escaping, which should work with any input string.

Also note how variable $lineEscaped is only referenced once, in the regex portion of the s command, whereas the substitution-string portion simply references what the regex matched (which avoids the need to escape the variable again, using different rules):
& in the substitution string represents the entire match, and \1 the first capture group (parenthesized subexpression, $...$).

For simplicity, the second sed command uses double quotes in order to embed the value of shell variable $lineEscaped in the sed script, but it is generally preferable to use single-quoted scripts so as to avoid confusion between what the shell interprets up front vs. what sed ends up seeing.

For instance, $ is special to both the shell and sed, and in the above script the end-of-line anchor $ in the sed regex must therefore be escaped as \$ to prevent the shell from interpreting it.
One way to avoid confusion is to selectively splice double-quoted shell-variable references into the otherwise single-quoted script:

sed -i 's/^'"$lineEscaped"'$/#&/' file.txt

`awk` solution

awk offers literal string matching, which obviates the need for escaping:

awk -v line="$line" '$0 == line { $0 = "#" $0 } 1' file.txt > $$.tmp && mv $$.tmp file.txt

^{If you have GNU Awk v4.1+, you can use -i inplace for in-place updating.}

And the reverse (un-commenting):

awk -v line="#$line" '$0 == line { $0 = substr($0, 2) } 1' file.txt > $$.tmp && 
  mv $$.tmp file.txt

@Swifting My pleasure; glad it helped; I amended the question's title and tags a little to make the question easier to discover by future readers. — mklement0, Jan 23 '17 at 12:57

Escaping a variable with special characters within sed - comment and uncomment an arbitrary line of source code

2 Answers2

sed solution

awk solution

`sed` solution

`awk` solution