Separately redirecting and recombining stderr/stdout without losing ordering

Question

I want to execute a command and want to redirect stderr and stdout as below:

stderr and stdout -> should be written ONLY to logs.log file while keeping the order

stderr -> should be printed to SCREEN and also written to errors.log

So far I can redirect them both to screen and the file log.txt like this:

command 2>&1 | tee logs.log

But the above is not what I need.

To make it more clear again what the results needs to be.

After command is executed I need to see on screen only the result of stderr, I need to have one file named errors.log with stderr, and I need to have another file named logs.log with the result of both stdout and stderr in the original order in which they were created.

Separate redirections while maintaining order is literally not possible: The operating system doesn't provide the guarantees you'd need to have to support it. — Charles Duffy, Aug 18 '17 at 16:30
Well, not possible unless you rely on instrumenting syscalls a la reconstructing output from sysdig or strace. — Charles Duffy, Aug 18 '17 at 16:30
What *is* the command you're running, by the way? You might be able to get the guarantees want if it's written in a language with a logging framework you can configure to do all the necessary writes as a blocking operation. — Charles Duffy, Aug 18 '17 at 16:44
BTW, you're referring to `logs.log`, `errors.log` and `log.txt` -- do you really have *three* intended output files, additional to the console? — Charles Duffy, Aug 18 '17 at 17:03
@CharlesDuffy I edited i have only 2, log.txt and logs.log are the same. Question is edited now — Kristi Jorgji, Aug 21 '17 at 07:19

Charles Duffy · Accepted Answer · 2018-10-13T18:14:40.027

Preserving perfect order while performing separate redirections is not even theoretically possible without some ugly hackery. Ordering is only preserved in writes (in O_APPEND mode) directly to the same file; as soon as you put something like tee in one process but not the other, ordering guarantees go out the window and can't be retrieved without keeping information about which syscalls were invoked in what order.

So, what would that hackery look like? It might look something like this:

# eat our initialization time *before* we start the background process
sudo sysdig-probe-loader

# now, start monitoring syscalls made by children of this shell that write to fd 1 or 2
# ...funnel content into our logs.log file
sudo sysdig -s 32768 -b -p '%evt.buffer' \
  "proc.apid=$$ and evt.type=write and (fd.num=1 or fd.num=2)" \
  > >(base64 -i -d >logs.log) \
  & sysdig_pid=$!

# Run your-program, with stderr going both to console and to errors.log
./your-program >/dev/null 2> >(tee errors.log)

That said, this remains ugly hackery: It only catches writes direct to FDs 1 and 2, and doesn't track any further redirections that may take place. (This could be improved by performing the writes to FIFOs, and using sysdig to track writes to those FIFOs; that way fdup() and similar operations would work as-expected; but the above suffices to prove the concept).

Making Separate Handling Explicit

Here we demonstrate how to use this to colorize only stderr, and leave stdout alone -- by telling sysdig to generate a stream of JSON as output, and then iterating over that:

exec {colorizer_fd}> >(
  jq --unbuffered --arg startColor "$(tput setaf 1)" --arg endColor "$(tput sgr0)" -r '
    if .["fd.filename"] == "stdout" then
      ("STDOUT: " + .["evt.buffer"])
    else
      ("STDERR: " + $startColor + .["evt.buffer"] + $endColor)
    end
  '
)

sudo sysdig -s 32768 -j -p '%fd.filename %evt.buffer' \
  "proc.apid=$$ and evt.type=write and proc.name != jq and (fd.num=1 or fd.num=2)" \
  >&$colorizer_fd \
  & sysdig_pid=$!

# Run your-program, with stdout and stderr going to two separately-named destinations
./your-program >stdout 2>stderr

Because we're keying off the output filenames (stdout and stderr), these need to be constant for the above code to work -- any temporary directory desired can be used.

Obviously, you shouldn't actually do any of this. Update your program to support whatever logging infrastructure is available in its native language (Log4j in Java, the Python logging module, etc) to allow its logging to be configured explicitly.

Yes this has lot of overhead but thanks for the suggestion. The program is not mine I just want to log the following way which is complicated and not an usual case. Was just curious of what possible solutions are there to achieve that — Kristi Jorgji, Aug 22 '17 at 10:07

score 2 · Answer 2 · answered Aug 18 '17 at 16:26

2

This will get you most of the way there:

your_command 2> >(tee -a logs.log errors.log) 1>>logs.log

but I don't think you'll be able to exactly preserve the order of the output in the logs.log file.

answered Aug 18 '17 at 16:26

glenn jackman

238,783
38
220
352

Separately redirecting and recombining stderr/stdout without losing ordering

2 Answers2

Making Separate Handling Explicit

Linked

Related