How to assign a heredoc value to a variable in Bash?

Question

I have this multi-line string (quotes included):

abc'asdf"
$(dont-execute-this)
foo"bar"''

How would I assign it to a variable using a heredoc in Bash?

I need to preserve newlines.

I don't want to escape the characters in the string, that would be annoying...

Answer 1

You can avoid a useless use of cat and handle mismatched quotes better with this:

$ read -r -d '' VAR <<'EOF'
abc'asdf"
$(dont-execute-this)
foo"bar"''
EOF

If you don't quote the variable when you echo it, newlines are lost. Quoting it preserves them:

$ echo "$VAR"
abc'asdf"
$(dont-execute-this)
foo"bar"''

If you want to use indentation for readability in the source code, use a dash after the less-thans. The indentation must be done using only tabs (no spaces).

$ read -r -d '' VAR <<-'EOF'
    abc'asdf"
    $(dont-execute-this)
    foo"bar"''
    EOF
$ echo "$VAR"
abc'asdf"
$(dont-execute-this)
foo"bar"''

If, instead, you want to preserve the tabs in the contents of the resulting variable, you need to remove tab from IFS. The terminal marker for the here doc (EOF) must not be indented.

$ IFS='' read -r -d '' VAR <<'EOF'
    abc'asdf"
    $(dont-execute-this)
    foo"bar"''
EOF
$ echo "$VAR"
    abc'asdf"
    $(dont-execute-this)
    foo"bar"''

Tabs can be inserted at the command line by pressing Ctrl-V Tab. If you are using an editor, depending on which one, that may also work or you may have to turn off the feature that automatically converts tabs to spaces.

Answer 2

Use $() to assign the output of cat to your variable like this:

VAR=$(
cat <<'END_HEREDOC'
abc'asdf"
$(dont-execute-this)
foo"bar"''
END_HEREDOC
)

# this will echo variable with new lines intact
echo "$VAR"
# this will echo variable without new lines (changed to space character)
echo $VAR

Making sure to delimit starting END_HEREDOC with single-quotes. This will prevent the content of the heredoc from being expanded, so dont-execute-this will not be executed.

Note that ending heredoc delimiter END_HEREDOC must be alone on the line (hence the ending parenthesis ) is on the next line).

Thanks to @ephemient for the answer.

Answer 3

this is variation of Dennis method, looks more elegant in the scripts.

function definition:

define(){ IFS='\n' read -r -d '' ${1} || true; }

usage:

define VAR <<'EOF'
abc'asdf"
$(dont-execute-this)
foo"bar"''
EOF

echo "$VAR"

enjoy

p.s. made a 'read loop' version for shells that do not support read -d. should work with set -eu and unpaired backticks, but not tested very well:

define(){ o=; while IFS="\n" read -r a; do o="$o$a"'
'; done; eval "$1=\$o"; }

Answer 4

VAR=<<END
abc
END

doesn't work because you are redirecting stdin to something that doesn't care about it, namely the assignment

export A=`cat <<END
sdfsdf
sdfsdf
sdfsfds
END
` ; echo $A

works, but there's a back-tic in there that may stop you from using this. Also, you should really avoid using backticks, it's better to use the command substitution notation $(..).

export A=$(cat <<END
sdfsdf
sdfsdf
sdfsfds
END
) ; echo $A

Answer 5

There is still no solution that preserves newlines.

This is not true - you're probably just being misled by the behaviour of echo:

echo $VAR # strips newlines

echo "$VAR" # preserves newlines

Answer 6

Branching off Neil's answer, you often don't need a var at all, you can use a function in much the same way as a variable and it's much easier to read than the inline or read-based solutions.

$ complex_message() {
  cat <<'EOF'
abc'asdf"
$(dont-execute-this)
foo"bar"''
EOF
}

$ echo "This is a $(complex_message)"
This is a abc'asdf"
$(dont-execute-this)
foo"bar"''

Answer 7

An array is a variable, so in that case mapfile will work

mapfile y <<'z'
abc'asdf"
$(dont-execute-this)
foo"bar"''
z

Then you can print like this

printf %s "${y[@]}"

Answer 8

I can't believe I'm the first to post this.

@Erman and @Zombo are close, but mapfile doesn't just read arrays...

Consider this:

#!/bin/bash
mapfile -d '' EXAMPLE << 'EOF'
Hello
こんにちは
今晩は
小夜なら
EOF
echo -n "$EXAMPLE"

Yielding:

Hello
こんにちは
今晩は
小夜なら

$'' is the delimiter given to mapfile, it will never occur, it means "not delimited".

So there's no need for a useless use of cat and no need to incur the penalty of recombining arrays.

Furthermore, you get this benefit:

$ echo $EXAMPLE
Hello こんにちは 今晩は 小夜なら

Which you do not receive with @Zombo's method:

mapfile y <<'z'
abc'asdf"
$(dont-execute-this)
foo"bar"''
z
echo $y

abc'asdf"

Bonus

If you run it through head -c -1 you can also get rid of that last newline in a way that won't be non-performant:

unset EXAMPLE
mapfile -d '' EXAMPLE < <(head -c -1 << EOF
Hello
こんにちは
今晩は
小夜なら
EOF
)
printf "%q" "$EXAMPLE"

$'Hello\nこんにちは\n今晩は\n小夜なら'

Answer 9

assign a heredoc value to a variable

VAR="$(cat <<'VAREOF'
abc'asdf"
$(dont-execute-this)
foo"bar"''
VAREOF
)"

used as an argument of a command

echo "$(cat <<'SQLEOF'
xxx''xxx'xxx'xx  123123    123123
abc'asdf"
$(dont-execute-this)
foo"bar"''
SQLEOF
)"

Answer 10

Thanks to dimo414's answer, this shows how his great solution works, and shows that you can have quotes and variables in the text easily as well:

example output

$ ./test.sh

The text from the example function is:
  Welcome dev: Would you "like" to know how many 'files' there are in /tmp?

  There are "      38" files in /tmp, according to the "wc" command

test.sh

#!/bin/bash

function text1()
{
  COUNT=$(\ls /tmp | wc -l)
cat <<EOF

  $1 Would you "like" to know how many 'files' there are in /tmp?

  There are "$COUNT" files in /tmp, according to the "wc" command

EOF
}

function main()
{
  OUT=$(text1 "Welcome dev:")
  echo "The text from the example function is: $OUT"
}

main

Answer 11

I found myself having to read a string with NULL in it, so here is a solution that will read anything you throw at it. Although if you actually are dealing with NULL, you will need to deal with that at the hex level.

$ cat > read.dd.sh

read.dd() {
     buf= 
     while read; do
        buf+=$REPLY
     done < <( dd bs=1 2>/dev/null | xxd -p )

     printf -v REPLY '%b' $( sed 's/../ \\\x&/g' <<< $buf )
}

Proof:

$ . read.dd.sh
$ read.dd < read.dd.sh
$ echo -n "$REPLY" > read.dd.sh.copy
$ diff read.dd.sh read.dd.sh.copy || echo "File are different"
$ 

HEREDOC example (with ^J, ^M, ^I):

$ read.dd <<'HEREDOC'
>       (TAB)
>       (SPACES)
(^J)^M(^M)
> DONE
>
> HEREDOC

$ declare -p REPLY
declare -- REPLY="  (TAB)
      (SPACES)
(^M)
DONE

"

$ declare -p REPLY | xxd
0000000: 6465 636c 6172 6520 2d2d 2052 4550 4c59  declare -- REPLY
0000010: 3d22 0928 5441 4229 0a20 2020 2020 2028  =".(TAB).      (
0000020: 5350 4143 4553 290a 285e 4a29 0d28 5e4d  SPACES).(^J).(^M
0000030: 290a 444f 4e45 0a0a 220a                 ).DONE

Answer 12

Here's a way to do it that is (imho) quite elegant and avoids a UUOC:

  VAR=$(sed -e 's/[ ]*\| //g' -e '1d;$d' <<'--------------------'
      | 
      | <!DOCTYPE html>
      | <html>
      |   <head>
      |     <script src='script.js'></script>
      |   </head>
      |   <body>
      |     <span id='hello-world'></span>
      |   </body>
      | </html>
      | 
--------------------
    )

The '|' characters define the margin, and only the whitespace to the right of the margin is respected in the printed string. The '1d;$d' strips the first and last line, which are just added as a top and bottom margin around the content. Everything can be indented to whatever level you like, except the HEREDOC delimiter, which in this case is just a bunch of hyphens.

echo "$VAR"

# prints

<!DOCTYPE html>
<html>
  <head>
    <script src='script.js'></script>
  </head>
  <body>
    <span id='hello-world'></span>
  </body>
</html>

Answer 13

$TEST="ok"
read MYTEXT <<EOT
this bash trick
should preserve
newlines $TEST
long live perl
EOT
echo -e $MYTEXT