How to replace a hashtag curly bracket string with an environment variable by using sed

Question

I have being trying to write a bash script that can search recursively in a directory and replace multiple strings e.g. #{DEMO_STRING_1} etc with an environment variable e.g. $sample1.

Full script:

#!/bin/sh

find /my/path/here -type f -name '*.js' -exec sed -i \
    -e 's/#{DEMO_STRING_1}/'"$sample1"'/g' \
    -e 's/#{DEMO_STRING_2}/'"$sample2"'/g' \
    -e 's/#{DEMO_STRING_3}/'"$sample3"'/g' \
    -e 's/#{DEMO_STRING_4}/'"$sample4"'/g' \
    -e 's/#{DEMO_STRING_5}/'"$sample5"'/g' \
    -e 's/#{DEMO_STRING_6}/'"$sample6"'/g' \
    -e 's/#{DEMO_STRING_7}/'"$sample7"'/g' \
    -e 's/#{DEMO_STRING_8}/'"$sample8"'/g' \
    {} +

I can not figure out how to replace strings with hashtag with curly brackets.

I tried this example: sed find and replace with curly braces or Environment variable substitution in sed but I can not figure out how to combine them.

What I am missing? I searched also for characters that need to be escaped e.g. What characters do I need to escape when using sed in a sh script? but again not the characters that I need.

The specific format is throwing the following error:

sed: bad option in substitution expression

Where am I going so wrong?

Update: Sample of environment variables:

1. https://www.example.com
2. /sample string/
3. 12345-abcd-54321-efgh
4. base64 string

All the cases above are environment variables that I would like to replace. All environment variables are within double quotes.

Answer 1

It is important to understand that the environment variable references are expanded by the shell, as it prepares to execute the command, not by the command itself (sed in this case). The command sees only the results of the expansions.

In your case, that means that if any of the environment variables' values contain characters that are meaningful to sed in context, such as unescaped (to sed) slashes (/), then sed will attribute special significance to them instead of interpreting them as ordinary characters. For example, given a sed command such as

sed -e "s/X/${var}/" <<EOF
Replacement:  X
EOF

, if the value of $var is Y then the output will be

Replacement: Y

, but if the value of $var is /path/to/Y then sed will fail with the same error you report. This happens because the sed command actually run is the same as if you had typed

sed -e s/X//path/to/Y

, which contains an invalid s instruction. Probably the best alternative would be to escape the replacement-string characters that otherwise would be significant to sed. You can do that by interposing a shell function:

escape_replacement() {
  # replace all \ characters in the first argument with double backslashes.
  # Note that in order to do that here, we need to escape them from the shell
  local temp=${1//\\/\\\\}

  # Replace all & characters with \&
  temp=${temp//&/\\&}

  # Replace all / characters with \/, and write the result to standard out.
  # Use printf instead of echo to avoid edge cases in which the value to print
  # is interpreted to be or start with an option.
  printf -- "%s" "${temp//\//\\/}"
}

Then the script would use it like this:

find /my/path/here -type f -name '*.js' -exec sed -i \
    -e 's/#{DEMO_STRING_1}/'"$(escape_replacement "$sample1")"'/g' \
...

Note that you probably also want to use a shebang line that explicitly specifies a shell that supports substitution references (${parameter/pattern/replacement}), because these are not required by POSIX, and you might run into a system where /bin/sh is a shell that does not support them. If you're willing to rely on Bash then that should be reflected in your shebang line. Alternatively, you could prepare a version of the escape_replacement function that does not rely on substitution references.

Answer 2

If you use perl - you don't need to escape anything.

With your shell variable exported you can access it via $ENV{name} inside perl.

examples:

samples=(
    https://www.example.com
    '/sample string/'
    12345-abcd-54321-efgh
    'base64 string'
    $'multi\nline'
)

for sample in "${samples[@]}"
do
    echo '---'
    export sample
    echo 'A B #{DEMO_STRING_1} C' |
        perl -pe 's/#{DEMO_STRING_1}/$ENV{sample}/g'
done
echo '---'

Output:

---
A B https://www.example.com C
---
A B /sample string/ C
---
A B 12345-abcd-54321-efgh C
---
A B base64 string C
---
A B multi
line C
---

To add the -i option you can: perl -pi -e 's///'