how to replace a variable in shell script string

Question

I'm having problems getting this to work...

I have a variable that is holding a SQL to with a placeholder:

echo $SQL
SELECT PX_PROMOTION_ID, PRIORITY, STATUS, EXCLSVE, TYPE, PERORDLMT, PERSHOPPERLMT, TOTALLMT, RSV_INT, PX_GROUP_ID, CAMPAIGN_ID, STOREENT_ID, VERSION, REVISION, EFFECTIVE, TRANSFER, CDREQUIRED, EXPIRE, LASTUPDATEBY, TO_CHAR(LASTUPDATE, 'YYYYMMDD HH24MMSS') AS LASTUPDATE, TO_CHAR(STARTDATE, 'YYYYMMDD HH24MMSS') AS STARTDATE, TO_CHAR(ENDDATE, 'YYYYMMDD HH24MMSS') AS ENDDATE, TO_CHAR(RSV_TIME, 'YYYYMMDD HH24MMSS') AS RSV_TIME, RSV_REAL, TGTSALES, NAME, CODE, RSV_VCH, OPTCOUNTER FROM PX_PROMOTION WHERE LASTUPDATE BETWEEN (SELECT MAX(BATCHSTART) FROM XRPTEBATCHCONTROL) AND TIMESTAMP('$BATCH_END')

I have another variable that holds the value:

echo $BATCH_END
2012-11-14 17:06:13

I want to replace the placeholder with the value. I'm not particularly great at Unix scripting, but I've tried this:

echo $SQL | sed -e "s/'$BATCH_END/$BATCH_END/g"

but it still doesn't get replaced...

Can anyone help? I want to replace the placeholder, and keep the final string assigned to $SQL

I also need to know how to get the value of the output back into the variable, for example, I tried:

 SQL=`echo "$SQL" | echo "${SQL//\$BATCH_END/$BATCH_END}"`

Too much echo. Use `SQL="${SQL//\$BATCH_END/$BATCH_END}"` instead — doubleDown, Nov 14 '12 at 07:49
[Here](http://stackoverflow.com/q/2914220/619001) is a more general version of this question, covering replacement of all variables without knowing in advance what the variables will be named. — Mark Haferkamp, Jun 07 '15 at 09:49

sampson-chen · Accepted Answer · 2012-11-14T08:03:24.520

49

You are missing the end of that single-quote pair in your script.

Change from:

echo $SQL | sed -e "s/'$BATCH_END/$BATCH_END/g"

To:

echo $SQL | sed -e "s/\$BATCH_END/$BATCH_END/g"

Updated - as per followup comment:

To save the result of the above replacement back into $SQL, do either of the following:

# Preferred way
SQL=$(echo $SQL | sed -e "s/\$BATCH_END/$BATCH_END/g")

# Old way
SQL=`echo $SQL | sed -e "s/\$BATCH_END/$BATCH_END/g"`

This is called command substitution. Either syntax ($(...) vs. enclosure by backticks) works, but the preferred one allows you to do nesting.

The preferred-preferred way: Herestring

This is probably a bit more advanced than what you care about, but doing it in the following way will save you a subprocess from having to use echo unnecessarily:

SQL=$(sed -e "s/\$BATCH_END/$BATCH_END/g" <<< $SQL)

edited Nov 14 '12 at 08:03

answered Nov 14 '12 at 06:25

sampson-chen

45,805
12
84
81

The replacement value still needs quotes around it. – Jonathan Leffler Nov 14 '12 at 06:27
ok I got this far:SQL=echo $SQL | sed -e "s/\$BATCH_END/$BATCH_END/g", but the assignment back is not working although the SED seems to be now. How do I assign it back to the SQL variable? – Richard G Nov 14 '12 at 06:34
@RichardG See my updated answer for assigning the replaced result back into the SQL variable, I think that is what you are looking for. – sampson-chen Nov 14 '12 at 07:53

score 3 · Answer 2 · answered Nov 14 '12 at 06:32

In my terminal:

$ SQL="SELECT PX_PROMOTION_ID, PRIORITY, STATUS, EXCLSVE, TYPE, PERORDLMT, PERSHOPPERLMT, TOTALLMT, RSV_INT, PX_GROUP_ID, CAMPAIGN_ID, STOREENT_ID, VERSION, REVISION, EFFECTIVE, TRANSFER, CDREQUIRED, EXPIRE, LASTUPDATEBY, TO_CHAR(LASTUPDATE, 'YYYYMMDD HH24MMSS') AS LASTUPDATE, TO_CHAR(STARTDATE, 'YYYYMMDD HH24MMSS') AS STARTDATE, TO_CHAR(ENDDATE, 'YYYYMMDD HH24MMSS') AS ENDDATE, TO_CHAR(RSV_TIME, 'YYYYMMDD HH24MMSS') AS RSV_TIME, RSV_REAL, TGTSALES, NAME, CODE, RSV_VCH, OPTCOUNTER FROM PX_PROMOTION WHERE LASTUPDATE BETWEEN (SELECT MAX(BATCHSTART) FROM XRPTEBATCHCONTROL) AND TIMESTAMP('\$BATCH_END')"
$ # (observe: I escaped the $ sign to have the same variable as you)
$ echo "$SQL"
SELECT PX_PROMOTION_ID, PRIORITY, STATUS, EXCLSVE, TYPE, PERORDLMT, PERSHOPPERLMT, TOTALLMT, RSV_INT, PX_GROUP_ID, CAMPAIGN_ID, STOREENT_ID, VERSION, REVISION, EFFECTIVE, TRANSFER, CDREQUIRED, EXPIRE, LASTUPDATEBY, TO_CHAR(LASTUPDATE, 'YYYYMMDD HH24MMSS') AS LASTUPDATE, TO_CHAR(STARTDATE, 'YYYYMMDD HH24MMSS') AS STARTDATE, TO_CHAR(ENDDATE, 'YYYYMMDD HH24MMSS') AS ENDDATE, TO_CHAR(RSV_TIME, 'YYYYMMDD HH24MMSS') AS RSV_TIME, RSV_REAL, TGTSALES, NAME, CODE, RSV_VCH, OPTCOUNTER FROM PX_PROMOTION WHERE LASTUPDATE BETWEEN (SELECT MAX(BATCHSTART) FROM XRPTEBATCHCONTROL) AND TIMESTAMP('$BATCH_END')
$ BATCH_END="2012-11-14 17:06:13"
$ echo "$BATCH_END"
2012-11-14 17:06:13
$ # Now the replacement:
$ echo "${SQL//\$BATCH_END/$BATCH_END}"
SELECT PX_PROMOTION_ID, PRIORITY, STATUS, EXCLSVE, TYPE, PERORDLMT, PERSHOPPERLMT, TOTALLMT, RSV_INT, PX_GROUP_ID, CAMPAIGN_ID, STOREENT_ID, VERSION, REVISION, EFFECTIVE, TRANSFER, CDREQUIRED, EXPIRE, LASTUPDATEBY, TO_CHAR(LASTUPDATE, 'YYYYMMDD HH24MMSS') AS LASTUPDATE, TO_CHAR(STARTDATE, 'YYYYMMDD HH24MMSS') AS STARTDATE, TO_CHAR(ENDDATE, 'YYYYMMDD HH24MMSS') AS ENDDATE, TO_CHAR(RSV_TIME, 'YYYYMMDD HH24MMSS') AS RSV_TIME, RSV_REAL, TGTSALES, NAME, CODE, RSV_VCH, OPTCOUNTER FROM PX_PROMOTION WHERE LASTUPDATE BETWEEN (SELECT MAX(BATCHSTART) FROM XRPTEBATCHCONTROL) AND TIMESTAMP('2012-11-14 17:06:13')

Done!

ok that's a neat approach rather than SED, but how do I get it assigned back to the SQL variable, tried: SQL=`echo "${SQL//\$BATCH_END/$BATCH_END}"` — Richard G, Nov 14 '12 at 06:39
You should be able to use: `SQL=${SQL//\$BATCH_END/$BATCH_END}` just like that. You might need double quotes around the RHS of the assigment in some shells, but not in `bash` (and only `bash`, AFAIK, supports the `//` modifier on parameter expansion — though it wouldn't surprise me to find that `zsh` does too). — Jonathan Leffler, Nov 14 '12 at 08:08
Great listing all used commands! Helped to understand what's going on. — Bruno Bieri, Oct 12 '16 at 07:05
how can the same be achieved in ksh. the same script says bad substitution — sjd, May 09 '18 at 07:40

kmkaplan · Answer 3 · 2012-11-14T06:56:08.007

1

You need to quote the first $ so that it does not get expanded as a shell variable.

echo "$SQL" | sed -e "s/'\$BATCH_END'/'$BATCH_END'/g"

… Or choose an easier placeholder, something like @BATCH_END@ for instance.

To assign the result back to $SQL you will need some more shell escaping:

SQL=`echo "$SQL" | sed -e "s/'\\\$BATCH_END'/'$BATCH_END'/g"`

edited Nov 14 '12 at 06:56

answered Nov 14 '12 at 06:32

kmkaplan

18,655
4
51
65

1

Strongly recommend `SQL=$(...)` over ``SQL=`...` ``. – Jonathan Leffler Nov 14 '12 at 08:03

score 0 · Answer 4 · answered Nov 14 '12 at 06:32

One way to do it is with 'differential quoting' in a single argument:

echo "$SQL" | sed -e 's/$BATCH_END/'"$BATCH_END/g"

The first part of the -e option is in single quotes, so the shell does not expand the first $BATCH_END and it can match the word in the SQL statement. The second part is in double quotes, so the shell expands the second $BATCH_END and places its text into the SQL.

If you needed to worry about the single quotes around $BATCH_END, you'd have to play other tricks; probably a backslash would be simplest (and it's a viable option anyway):

echo "$SQL" | sed -e "s/'\$BATCH_END'/'$BATCH_END'/g"

The backslash stops the shell expanding the first $BATCH_END but the absence of a backslash means the second is expanded. Inside the double quotes, the single quotes lose their 'no expansion' property.

Brian Campbell · Answer 5 · 2012-11-14T07:08:15.800

The problem is that you are using a double-quoted string in the shell. In a double quoted string, variables like $BATCH_END get interpreted as shell variables and interpolated. The ' character has no special meaning within a double quoted string; it doesn't prevent variables from being interpolated. So your $BATCH_END string is being substituted in both places; your sed invocation is equivalent to:

sed -e "s/'2012-11-14 17:06:13/2012-11-14 17:06:13/"

Which as you can see, is not very helpful (you also have a stray ' in there). You need to escape the $ sign, to prevent it from being interpreted as a shell variable:

sed -e "s/\$BATCH_END/$BATCH_END/"

score 0 · Answer 6 · edited Nov 10 '20 at 10:37

0

Situation to add 80% memory in "maxmemory" in redis.conf:

# GET TOTAL MEMORY
totalmemory=$(awk '{ printf "%.2f", $2/1024 ; exit}' /proc/meminfo | awk -F"." '{print $1}');

# CALCUL
ramredis=$(echo $totalmemory/100*80 | bc);

# APPLY
sed -i 's/#maxmemory/maxmemory x mb/g' /etc/redis.conf;sed -i 's/ x /'\ $ramredis'/g' /etc/redis.conf;

edited Nov 10 '20 at 10:37

dmigo

2,849
4
41
62

answered Nov 09 '20 at 21:02

Thiago Silva

11
1

score 0 · Answer 7 · answered Jan 20 '22 at 11:06

0

Use double quote

sed -i "s/targetString/$newName/g" targetFile.toml

answered Jan 20 '22 at 11:06

Russo

2,186
2
26
42

how to replace a variable in shell script string

7 Answers7