6

Content of test.csv is:

XYZ,IN123

Here is my script:

Var1=IN123
Var2=A&B
sed -i "s/$Var1/$Var2/g" test.csv

This is my simple code to replace the content of test.csv, when the code finds IN123, it will be replaced by A&B.

So expected output is:

XYZ,A&B

But with the above code I am getting:

XYZ,AIN123B

What am I doing wrong?

anubhava
  • 761,203
  • 64
  • 569
  • 643
Programmer
  • 329
  • 2
  • 6
  • 25

4 Answers4

14

Since the question asks about how this can be done with SED, there is a less known feature about it that can be very useful here - the delimiters can be whatever you want as long as they are not in your replacement string.

So for the above you could do

sed -i "s|$Var1|$Var2|g" test.csv

..and can replace it with $ or : or anything else you like.

panksdmz
  • 421
  • 1
  • 5
  • 8
  • I wish I could give you +10, this helped me the most! Now sed looks clean and nice to me! Big thanks to you! – kamil_debski May 24 '21 at 07:05
  • But... Correct me if I'm wrong this doesn't work on every sed statement like: ```sed '/pattern/i line1' file.txt``` but in the modifed version it will NOT work: ```sed ':pattern:i line1' file.txt``` – kamil_debski May 24 '21 at 08:06
  • on MacOS, it show "sed: 1: invalid command code f" – Binh Thien Apr 23 '22 at 12:35
7

Unfortunately you have been caught out by the fact that & has a special meaning in a sed replacement string. It is a metacharacter that means "the entire pattern that was matched". In order to get a literal ampersand, you will have to escape it:

Var1='IN123'
Var2='A\&B'
sed -i "s/$Var1/$Var2/g" test.csv

The single quotes are necessary to prevent the \& from being interpreted as an escape sequence in the assignment. I added them to both variables for symmetry.

Tom Fenech
  • 72,334
  • 12
  • 107
  • 141
  • 1
    Hi Tom, This was a sample content which I posted. I am actually doing this from two input files. First file has Var1,Var2 and second file is the same test.csv. Do you mean in the input file itself, I have to handle this? I need to replace "&" with "\&" ? – Programmer Feb 19 '15 at 08:15
  • @JavaProgrammer if you want to use sed, then yes. However , given the full details of your problem it sounds like there are better alternatives available. You should edit you question to show a sample of your files and your desired output. – Tom Fenech Feb 19 '15 at 08:20
  • I'm getting the following error when executing the command: `sed: 1: "test.csv": undefined label 'est.csv'` – Boris Nov 16 '22 at 15:20
  • @Boris I think this could be due to the behaviour of the `-i` switch in your version of sed https://stackoverflow.com/a/69955786/2088135 Try like `sed -i ''` or pass a non-empty string to create a backup file with that suffix. – Tom Fenech Nov 17 '22 at 07:50
  • Thank you, `sed -i ''` worked! – Boris Nov 17 '22 at 15:41
4

If you don't want to deal with regex and escaping use awk like this:

Var1='IN123'
Var2='A&B'
awk -v v1="$Var1" -v v2="$Var2" 'BEGIN{FS=OFS=","} $2==v1{$2=v2} 1' test.csv
XYZ,A&B
anubhava
  • 761,203
  • 64
  • 569
  • 643
  • Can you expand on this? Particularly...well, the entire `awk` statement. I see `-v` makes a variable available to the `BEGIN` block and that block is executed after the assignment. Then there is the FS and OFS (output field separator) but what's going on with this part: `$2==v1{$2=v2} 1` – harperville Jul 24 '19 at 19:07
  • `$2==v1` checks if 2nd column equals to `v1` and `{$2=v2}` sets 2nd column to `v2` – anubhava Jul 24 '19 at 19:30
  • Okay, I see. What's the `1` at the very end? – harperville Jul 25 '19 at 16:58
  • `1` is for printing data. – anubhava Jul 25 '19 at 17:32
  • I've checked this on Linux and Mac, this does not work as much as I wish it did. Perhaps something changed in AWK since this post, can anyone fix this? – Farley Mar 08 '22 at 20:32
  • It works pretty well for me on OSX and Linux awk – anubhava Mar 08 '22 at 20:34
1

& is a special char, hence, you need to use a \ Var2 needs to be specified in this way

Var2="A\&B"

Vargan
  • 1,277
  • 1
  • 11
  • 35
  • Hi Vargan, This was a sample content which I posted. I am actually doing this from two input files. First file has Var1,Var2 and second file is the same test.csv. Do you mean in the input file itself, I have to handle this? I need to replace "&" with "\&" ? – Programmer Feb 19 '15 at 08:16