Can I set BASH_XTRACEFD=1 without breaking command substitution?

Question

I have a bash script with output that I'm trying to debug by running bash with -x. I'd also like the output to be redirected to stdout instead of stderr (because the logging framework we use treats all stderr content as high-priority errors).

However, command substitutions in my code are failing because the lines logged by set -x are getting mixed in with the actual intended output, as shown in the below log.

How can I avoid this bug while directing set -x logs to stdout?

Example:

root@ee0d2037fdca:/# $(echo "ls")
bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
root@ee0d2037fdca:/# set -x 
root@ee0d2037fdca:/# export BASH_XTRACEFD=1
+ export BASH_XTRACEFD=1
+ BASH_XTRACEFD=1
root@ee0d2037fdca:/# $(echo "ls")
+ ++ echo ls ls
bash: ++: command not found
root@ee0d2037fdca:/#

The string that's prefixed is based on `PS4`; you can change it. That said, I don't at all understand *why* you would do this -- `$(echo "ls")` will be treating the *output from* `ls` as data to be executed every bit as much as it treats the logged/traced command that way as well. — Charles Duffy, Mar 05 '19 at 19:16
What's the real goal you're trying to accomplish by using stdout here? There's almost certainly a better way, even if it means just creating a separate copy of the original stdout. — Charles Duffy, Mar 05 '19 at 19:17
...note that per POSIX definition, stdout is only for *output*; "diagnostic" content, which these traces certainly are, is supposed to be routed to stderr. — Charles Duffy, Mar 05 '19 at 19:20
@CharlesDuffy We're doing this because we want to see the what our startup process looks like when our containers are started (set -x), and we use command substitution during the startup process, and DataDog sets all output to STDERR with an ERROR flag when I just want it to be INFO — chizou, Mar 05 '19 at 19:20
Personally, I would lean towards fixing that on the datadog end, rather than by changing your script's behavior. They certainly shouldn't be assuming that stderr only contains errors; it's defined as the appropriate place for *all* informational logging. — Charles Duffy, Mar 05 '19 at 19:21

Charles Duffy · Accepted Answer · 2021-07-14T15:53:40.270

Instead of assigning 1 to BASH_XTRACEFD, make a new file descriptor with a copy of FD 1, and point BASH_XTRACEFD to that.

exec {BASH_XTRACEFD}>&1
set -x
$(echo "ls")  # note that this is bad practice; see BashFAQ #50

That way your command substitutions change the real FD 1, but leave the copy of the original FD 1 alone.

Note that {BASH_XTRACEFD}>&1 -- with no leading $, and where BASH_XTRACEFD is not previously defined -- is deliberate and correct syntax! See https://stackoverflow.com/a/41620630/14122 describing this syntax, which requires bash 4.1 or later.

Can I set BASH_XTRACEFD=1 without breaking command substitution?

1 Answers1