Use anything as input to sed or Perl regex?

Question

If I do

comment () { sed -i "/$1/s/^/$2 /g" $3 }

comment /dev/sysmsg '#' /tmp/1
comment '*.err'     '#' /tmp/1

with input file

*.err;kern.notice;auth.notice   /dev/sysmsg
fff
ff

then it breaks, as / is used in sed as separator, and * also be treated as a regex.

Question

Is there a way to make it robust, so the input string in $1 can contain whatever I want? Or would I need to move to Perl?

https://stackoverflow.com/tags/sed/info page has couple of links related to this: https://stackoverflow.com/questions/5864146/use-slashes-in-sed-replace and https://stackoverflow.com/questions/29613304/is-it-possible-to-escape-regex-metacharacters-reliably-with-sed — Sundeep, Apr 12 '18 at 13:15
Yes, but it just moves the problem to another char I can't use. — Sandra Schlichting, Apr 12 '18 at 13:17
You can use [`\Q...\E`](http://perldoc.perl.org/functions/quotemeta.html) in Perl. If I understand your intentions correctly, try e.g. `function comment { ( perl -i -slpe 's/^(?=\Q$string\E)/$add/' -- -string="$1" -add="$2" $3 ) }` — haukex, Apr 12 '18 at 13:24

glenn jackman · Accepted Answer · 2018-04-12T14:05:35.447

Sure, use bash parameter substitution to escape the troublesome slash character:

comment () { sed -i "/${1//\//\\/}/ s/^/${2//\//\\/} /" "$3"; }

Notes:

You need to protect slashes in both the pattern and the replacement parts.
if your search is anchored, using "g" is pointless because the pattern can match at most once.
quote all your variables: if the filename contains a space, your code breaks.
one line functions require a semicolon before the closing brace.

Demo

$ cat file
test/1/
test/2/
test/3/

$ comment 'test/2' '//' file

$ cat file
test/1/
// test/2/
test/3/

I realized I'm not escaping regex special characters. The safest way is to escape any non-alphanumeric characters:

comment () { 
    local pattern=$(sed 's/[^[:alnum:]]/\\&/g' <<<"$1")
    local replace=${2//\//\\/}
    local file=$3
    sed -i "/$pattern/ s/^/$replace /" "$file"
}

But since you want to do plain text matching, sed is probably not the best tool:

comment() { 
    perl -i -pse '$_ = "$leader $_" if index($_, $search) > -1' -- \
        -search="$1" \
        -leader="$2" \
        "$3"
}

ikegami · Answer 2 · 2018-04-13T16:06:09.290

Best to avoid generating code from a shell script.

comment () {
   perl -i -pe'BEGIN { ($s,$r)=splice(@ARGV,0,2) }  $_ = "$r $_" if /\Q$s/' -- "$@"
}

or

comment () {
   s="$1" r="$2" perl -i -pe'$_ = "$ENV{r} $_" if /\Q$ENV{s}/' -- "$3"
}

or

comment () {
   perl -i -spe'$_ = "$r $_" if /\Q$s/' -- -s="$1" -r="$2" -- "$3"
}

Supports:

Arbitrary text for the search string (including characters that might normally be special in regex patterns, such as *). This is achieved by using quotemeta (as \Q) to convert the text into a regex pattern that matches that text.
Arbitrary file names (including those that contain spaces or start with -), thanks to proper quoting and the use of --.

Use anything as input to sed or Perl regex?

2 Answers2