why does $(cat) drop a newline

Question

foo.sh:

#!/bin/bash

x="$(cat)"

echo "got >>>$x<<<"

In the shell

#> echo abc | foo.sh
got >>>abc<<<

Where has the trailing newline character gone?

Answer 1

Bash command substitution operator intentionally drops trailing newlines; this is documented in the manual:

Bash performs the expansion by executing command in a subshell environment and replacing the command substitution with the standard output of the command, with any trailing newlines deleted.

This behavior is not specific to Bash, it was done by Bourne-descended shells before it. The reason is convenience - since most programs' output ends with a newline, retaining them would require code that processes the output to take the newline into account.

Since newlines are only removed from the end, one can work around this behavior by adding a dummy character at the end and stripping it manually:

# preserve trailing newlines
$ a="$(echo foo; echo .)"; a=${a:0:-1}
$ echo ">>>$a<<<"
>>>foo
<<<

# also works for programs that don't output a newline
$ b="$(printf foo; echo .)"; b=${b:0:-1}
$ echo ">>>$b<<<"
>>>foo<<<

Refer to this answer for more sophisticated workarounds.