0

Special case with ampersand (&) with sed

Want to replace string with & character string in bash

INPUT: 

var1=''XXX&XXX'
Input : sed -i 's/WW/$var1/g' test

Output:

$cat test
XXXWWXXX

expected output would be : XXX&XXX

PS: String would by Dynamic, So here i don't want to modify variable with escape character, Any solution in Shell ?

Nullpointer
  • 1,895
  • 20
  • 26

3 Answers3

2

Use it like this:

var1='XXX&XXX'

# Use double quotes in sed
# Note: & is escaped
sed "s/WW/${var1//&/\\&}/g" <<< '123 WW foo WW xyz.'

Output:

123 XXX&XXX foo XXX&XXX xyz.
anubhava
  • 761,203
  • 64
  • 569
  • 643
  • It will work But string would anything in anytime (It's a dynamic), So its not good idea to modify variable everytime. – Nullpointer Aug 27 '20 at 04:16
  • If you can see in first posted question where already mentioned that `here i don't want to modify variable with / character(slash), Any solution in Shell ?` may available in history. Thanks for you support...Keep help to community :) If you think, I posted answer is helpful for Users then verify/Accept it. – Nullpointer Aug 27 '20 at 07:12
  • I believe you mistyped `/` in your question and Ed commented same that `/` is not used for escaping but \ is – anubhava Aug 27 '20 at 07:47
1

Since you said i don't want to modify variable with / (sic) escape character - this will use literal string matches for both the existing text you want to match (WW) and its replacement (XXX&XXX) which is how you avoid escaping any chars in either of them:

var1='XXX&XXX'
old='WW' new="$var1" awk '
    BEGIN {
        old = ENVIRON["old"]
        new = ENVIRON["new"]
        lgth = length(old)
    }
    {
        head = ""
        tail = $0
        while ( s = index(tail,old) ) {
            head = head substr(tail,1,s-1) new
            tail = substr(tail,s+lgth)
        }
        $0 = head tail
        print
    }
' test

See How do I use shell variables in an awk script? for how I'm using the shell and awk variables.

You can't do the above with sed since sed doesn't understand literal strings so if you want to use sed then see Is it possible to escape regex metacharacters reliably with sed.

Ed Morton
  • 188,023
  • 17
  • 78
  • 185
  • 1
    Consider, Here String would be dynamic, So how can deal with `&` character. – Nullpointer Aug 27 '20 at 04:20
  • There are no Here Strings in your question so idk for sure what you're referring to. If it's just that the file `test` can be a Here String instead of a file then simply replace `test` with `<<< 'XXXWWXXX'`. Not sure why you had to ask about that - is there more to your question than that? – Ed Morton Aug 27 '20 at 13:54
0

Found the solution is that, In a recent shell this could be done like,

Need to put this //&/\\& static set of character to solve this ampersand (&) problem with sed

s1="XXX&XXX"

sed "s/ww/${s1//&/\\&}/g" <<< 'this is ww'

this is XXX&XXX

Other situation,

s2="&xx&xx&x&"

sed "s/ww/${s2//&/\\&}/g" <<< 'this is ww'

this is &xx&xx&x&

A quick search revealed the culprit;

The REPLACEMENT can contain \N (N being a number from 1 to 9, inclusive) 
references, which refer to the portion of the match which is contained 
between the Nth \( and its matching \). Also, the REPLACEMENT can contain 
unescaped & characters which reference the whole matched portion of the pattern space

Let’s try with awk

URI='https://user:pass@example.com/?num=42&type=magic'
$ echo 'my_link = %link%' | awk "{ gsub(/%link%/, \"$URI\"); print }"
my_link = https://user:pass@example.com/?num=42%link%type=magic

Find more detail on https://www.gnu.org/software/sed/manual/sed.html

Cheers!

Nullpointer
  • 1,895
  • 20
  • 26
  • 2
    You specifically said `i don't want to modify variable with / (sic) escape character` and here you are posting an answer that modifies the variable with an escape character. You're having to use double escapes and escape quotes because you're using double instead of single quotes around your scripts btw - don't do that and your programs will become simpler. Also, and importantly, note that `&` isn't the only character that'd need escaping it it appeared in `s1` - there are several. – Ed Morton Aug 27 '20 at 14:01