How to use variable to search and replace using sed or awk?

Question

I am trying to find and replace a value using variable with the help of sed and awk but not getting any output -

#cat temp.txt
this is Testing of date : 2016-11-25 07:20:10

It is printing the variable but not working in gsub function -

awk -v srch="2016-11-25 07:20:10" -v repl="[25/Nov/16:07:20:10]" '{print srch,repl}' temp.txt
2016-11-25 07:20:10 [25/Nov/16:07:20:10]

I tried below awk command -

awk -v srch="2016-11-25 07:20:10" -v repl="[25/Nov/16:07:20:10]" '{gsub("srch","repl",$0)}' temp.txt
awk -v srch="2016-11-25 07:20:10" -v repl="[25/Nov/16:07:20:10]" '{ gsub(srch,repl,$0)}' temp.txt
awk -v srch="2016-11-25 07:20:10" -v repl="[25/Nov/16:07:20:10]" '{ gsub("$srch","$repl",$0)}' temp.txt
awk -v srch="2016-11-25 07:20:10" -v repl="[25/Nov/16:07:20:10]" '{ gsub(srch,repl,$0)}' temp.txt
awk -v srch="2016-11-25 07:20:10" -v repl="[25/Nov/16:07:20:10]" '{print gsub(srch,repl,$0)}' temp.txt
awk -v srch="2016-11-25 07:20:10" -v repl="[25/Nov/16:07:20:10]" '{ gsub("srch","repl",$0)}' temp.txt
awk -v srch="2016-11-25 07:20:10" -v repl="[25/Nov/16:07:20:10]" '{ gsub("$srch","$repl",$0)}' temp.txt
awk -v srch="2016-11-25 07:20:10" -v repl="[25/Nov/16:07:20:10]" '{ gsub("$srch","$repl",$0)}' temp.txt
awk -v srch="2016-11-25 07:20:10" -v repl="[25/Nov/16:07:20:10]" '{ gsub(srch,repl,$0)}' temp.txt

#var1="2016-11-25 07:20:10"
#var2="[25/Nov/16:07:20:10]"
#echo $var1 $var4
2011-11-25 07:20:10 [25/Nov/11:07:20:10]
#awk -v srch="$var1" -v repl="$var4" '{ gsub(srch,repl,$0)}' temp.txt

With sed command -

#echo $var1 $var4
2011-11-25 07:20:10 [25/Nov/11:07:20:10]
sed 's/"$var1"/"$var4"/' temp.txt
sed 's/$var1/'"${var4}"'/' temp.txt
sed 's|$var1|'"${var4}"'|' temp.txt
sed 's/\$var1/${var4}/' temp.txt
sed 's/\$var1/$var4/' temp.txt
sed "s/"$var1"/"$var4"/" temp.txt
sed 's/'$var1'/'$var4'/' temp.txt
sed 's/'$var1'/'$var4\/' temp.txt
sed -e "s/${var1}/${var4}/' temp.txt
sed -e "s/${var1}/${var4}/" temp.txt
sed "s/$var1/$var4/" temp.txt
sed 's/'"$var1"'/'"$var4"'/' temp.txt
sed 's/'"$var1"'/'$var4'/' temp.txt

Not sure what i am missing.

Expected output -

#this is Testing of date : [25/Nov/11:07:20:10]

at least you tried something :) – Jean-François Fabre Nov 15 '16 at 18:09 — Jean-François Fabre, Nov 15 '16 at 18:09

score 2 · Answer 1 · answered Nov 15 '16 at 17:04

this one works, but you need to add print or nothing will be printed:

awk -v srch="2016-11-25 07:20:10" -v repl="[25/Nov/16:07:20:10]" '{ gsub(srch,repl,$0); print}' temp.txt

result:

this is Testing of date : [25/Nov/16:07:20:10]

with sed you can do it as well (as long as you use double quotes, or variables won't be evaluated), but since replacement string contains slashes, you have to change sed expression delimiter or you get a sed parsing error, I chose #

$ var1="2016-11-25 07:20:10"; var4="[25/Nov/11:07:20:10]"
$ sed "s#$var1#$var4#" temp.txt

result:

this is Testing of date : [25/Nov/11:07:20:10]

score 1 · Accepted Answer · edited May 23 '17 at 10:30

1

It would be:

var1="2016-11-25 07:20:10"
var2="[25/Nov/16:07:20:10]"
awk -v srch="$var1" -v repl="$var2" '{gsub(srch,repl)} 1' temp.txt
sed 's#'"$var1"'#'"$var2"'#g' temp.txt

You should also read Is it possible to escape regex metacharacters reliably with sed as the above would fail if var1 contained RE metacharacters or var2 contained backreferences.

edited May 23 '17 at 10:30

Community

1
1

answered Nov 15 '16 at 17:23

Ed Morton

188,023
17
78
185

score 0 · Answer 3 · answered Nov 15 '16 at 17:26

At a bit of a tangent - I would tackle it in perl and parse your date. (I know you didn't ask about perl, but I think it's a good fit for this - it's installed in most of the places you have sed or awk available)

It would work a bit like this:

#!/usr/bin/env perl
use strict;
use warnings;
use Time::Piece; 

#iterate __DATA__ special filehandle
while ( <DATA> ) {
    #extract time via regex
    my ( $time_str ) = m/([\d\-]+\s+[\d\:]+)$/; 
    next unless $time_str;
    #parse it into a Time::Piece object
    my $timestamp = Time::Piece -> strptime ("$time_str", "%Y-%m-%d %H:%M:%S");    
    #print it using a new format string
    my $new_time_str = $timestamp -> strftime("%d/%b/%y:%H:%M:%S");
    #replace old with new in line
    s/$time_str/\[$new_time_str\]/;
    #print current line
    print;
}

__DATA__
this is Testing of date : 2016-11-25 07:20:10

And for the sake of a command line solution, that condenses into:

perl -pe '($ts)=m/([\d\-]+\s+[\d\:]+)$/;s{$ts}{Time::Piece->strptime($ts,"%Y-%m-%d %H:%M:%S")->strftime("[%d/%b/%y:%H:%M:%S]")}e;'

You can pipe into this, or specify a filename as an argument, just like you would with sed or awk.

How to use variable to search and replace using sed or awk?

3 Answers3