Redirect stdout and stderr to Function

Question

I need help sending the output (stdin and stdout) from system commands to a bash function, while still accepting input from arguments. Something like the example that follows. Can someone point me down the right road?

LogMsg()
{
  DateTime=`date "+%Y/%m/%d %H:%M:%S"`
  echo '*****'$DateTime' ('$QMAKESPEC'): '$1 >> "$LogFile"
  echo $DateTime' ('$QMAKESPEC'): '$1
}

# Already works
LogMsg "This statement is sent directly"

# Wish I could do this:
# Capture both stdout & stderr of a system function to the logfile
# I do not presume that any of the syntax that follows is good
make 2>&1 >(LogMsg)

Do you still want the ability to call `LogMsg` with command-line arguments? — chepner, Aug 10 '12 at 16:42
All the answers use pipe, so `LogMsg` is only called after `make` has finished, and not in real-time as a hypothetic `>(LogMsg)` would. — pmiguelpinto90, Feb 10 '22 at 11:14

Lee Netherton · Accepted Answer · 2012-08-13T09:26:01.623

29

To do this you can use the read bash builtin:

LogMsg()
{
  read IN # This reads a string from stdin and stores it in a variable called IN
  DateTime=`date "+%Y/%m/%d %H:%M:%S"`
  echo '*****'$DateTime' ('$QMAKESPEC'): '$IN >> "$LogFile"
  echo $DateTime' ('$QMAKESPEC'): '$IN
}

And then use a pipe:

make 2>&1 | LogMsg

Update:

To be able to use stdin OR an argument as input (as per chepner's comment) you can do this:

LogMsg()
{
  if [ -n "$1" ]
  then
      IN="$1"
  else
      read IN # This reads a string from stdin and stores it in a variable called IN
  fi

  DateTime=`date "+%Y/%m/%d %H:%M:%S"`
  echo '*****'$DateTime' ('$QMAKESPEC'): '$IN >> "$LogFile"
  echo $DateTime' ('$QMAKESPEC'): '$IN
}

edited Aug 13 '12 at 09:26

answered Aug 10 '12 at 16:30

Lee Netherton

21,347
12
68
102

4

The only problem with this approach is that you can no longer call LogMsg without proving standard input. It's not clear if Ryan wants that flexibility. – chepner Aug 10 '12 at 16:46
Yes the flexibility is desired, I should have been more clear. – Ryan Aug 10 '12 at 16:58
Do you want to be able to *both*, i.e. `echo foo | LogMsg baz`? I've been unable to come up with a solution that can handle one, the other, or both. – chepner Aug 10 '12 at 16:59
Capturing the two strings is easy. But it is unclear what you would do with them. Print them one after the other? – Lee Netherton Aug 10 '12 at 17:02
No right now it is an either/or situation. I want to be able to capture an explicit argument OR capture the STDOUT & STDERR. – Ryan Aug 10 '12 at 17:04
@Lee yes, i figured arguments first (if present), then standard input (if present). But good to know that it's not necessary to support that scenario, and your solution will work. – chepner Aug 10 '12 at 17:13
I made a Gist of the script that I used to verify this was working correctly. https://gist.github.com/3315785 – Ryan Aug 10 '12 at 17:14
Helpful, thanks. Also discovered this: if you want to log a blank line, call `LogMsg " "`. If you omit the argument entirely, it hangs waiting for something to be supplied on stdin. – Mark Berry Jun 28 '14 at 19:08
@chepner This solution to a different question provides a good alternative syntax for using stdin or calling the function with an argument https://stackoverflow.com/a/35512655/12808884 – Leland Hepworth Aug 26 '20 at 20:20

score 6 · Answer 2 · answered Jul 27 '18 at 00:49

It's an old thread.. but I have used it to help me write a log function that will output also multiple lines of a command output:

# Defines function to grab a time stamp #
get_Time () { Time=$(date +%Y-%m-%d\ %H:%M:%S) ; }

write_Log()
{
get_Time
if [ -n "${1}" ]; then         # If it's from a "<message>" then set it
    IN="${1}"
    echo "${Time} ${IN}" | tee -a ${log_File}
else
    while read IN               # If it is output from command then loop it
    do
        echo "${Time} ${IN}" | tee -a ${log_File}
    done
fi
}

score 2 · Answer 3 · answered Apr 27 '20 at 01:11

Based on the previous answers, I put together some generic functions that work with or without a log file, as listed at the end of this post. These are handy for more complex scripts. I generally print terminal window messages to stderr so as to not interfere with legitimate program output that may need to be redirected. The functions can be called as follows:

scriptFolder=$(cd $(dirname "$0") && pwd)
scriptName=$(basename $scriptFolder)
# Start a log file that will be used by the logging functions
logFileStart ${scriptName} "${scriptFolder)/${scriptName}.log"

# The following logs the message string passed to the function.
# - use a space for empty lines because otherwise the logging function
#   will hang waiting for input
logInfo " "
logInfo "Starting to do some work."

# The following will log each 'stdout` and `stderr` line piped to the function.
someOtherProgram 2>&1 | logInfo

Functions...

# Echo to stderr
echoStderr() {
  # - if necessary, quote the string to be printed
  # - redirect stdout from echo to stderr
  echo "$@" 1>&2
  # Or, use an alternate echo such one that colors textT
  # ${echo2} "$@" 1>&2
}

# Print a DEBUG message
# - prints to stderr and optionally appends to log file if ${logFile} is defined globally
#   - see logFileStart() to start a log file
# - call with parameters or pipe stdout and stderr to this function: 2>&1 | logDebug
# - print empty lines with a space " " to avoid hanging the program waiting on stdin input
logDebug() {
  if [ -n "${1}" ]; then
    if [ -n "${logFile}" ]; then
      # Are using a log file
      echoStderr "[DEBUG] $@" 2>&1 | tee --append $logFile
    else
      # Are NOT using a log file
      echoStderr "[DEBUG] $@"
    fi
  else
    while read inputLine; do
      if [ -n "${logFile}" ]; then
        # Are using a log file
        echoStderr "[DEBUG] ${inputLine}" 2>&1 | tee --append $logFile
      else
        # Are NOT using a log file
        echoStderr "[DEBUG] ${inputLine}"
      fi
    done
  fi
}

# Print an ERROR message
# - prints to stderr and optionally appends to log file if ${logFile} is defined globally
#   - see logFileStart() to start a log file
# - call with parameters or pipe stdout and stderr to this function: 2>&1 | logError
# - print empty lines with a space " " to avoid hanging the program waiting on stdin input
logError() {
  if [ -n "${1}" ]; then
    if [ -n "${logFile}" ]; then
      # Are using a log file
      echoStderr "[ERROR] $@" 2>&1 | tee --append $logFile
    else
      # Are NOT using a log file
      echoStderr "[ERROR] $@"
    fi
  else
    while read inputLine; do
      if [ -n "${logFile}" ]; then
        # Are using a log file
        echoStderr "[ERROR] ${inputLine}" 2>&1 | tee --append $logFile
      else
        # Are NOT using a log file
        echoStderr "[ERROR] ${inputLine}"
      fi
    done
  fi
}

# Start a new logfile
# - name of program that is being run is the first argument
# - path to the logfile is the second argument
# - echo a line to the log file to (re)start
# - subsequent writes to the file using log*() functions will append
# - the global variable ${logFile} will be set for use by log*() functions
logFileStart() {
  local newLogFile now programBeingLogged
  programBeingLogged=$1
  # Set the global logfile, in case it was not saved
  if [ -n "${2}" ]; then
    logFile=${2}
  else
    # Set the logFile to stderr if not specified, so it is handled somehow
    logFile=/dev/stderr
  fi
  now=$(date '+%Y-%m-%d %H:%M:%S')
  # Can't use logInfo because it only appends and want to restart the file
  echo "Log file for ${programBeingLogged} started at ${now}" > ${logFile}
}

# Print an INFO message
# - prints to stderr and optionally appends to log file if ${logFile} is defined globally
#   - see logFileStart() to start a log file
# - call with parameters or pipe stdout and stderr to this function: 2>&1 | logInfo
# - print empty lines with a space " " to avoid hanging the program waiting on stdin input
logInfo() {
  if [ -n "${1}" ]; then
    if [ -n "${logFile}" ]; then
      # Are using a log file
      echoStderr "[INFO] $@" 2>&1 | tee --append $logFile
    else
      # Are NOT using a log file
      echoStderr "[INFO] $@"
    fi
  else
    while read inputLine; do
      if [ -n "${logFile}" ]; then
        # Are using a log file
        echoStderr "[INFO] ${inputLine}" 2>&1 | tee --append $logFile
      else
        # Are NOT using a log file
        echoStderr "[INFO] ${inputLine}"
      fi
    done
  fi
}

# Print an WARNING message
# - prints to stderr and optionally appends to log file if ${logFile} is defined globally
#   - see logFileStart() to start a log file
# - call with parameters or pipe stdout and stderr to this function: 2>&1 | logWarning
# - print empty lines with a space " " to avoid hanging the program waiting on stdin input
logWarning() {
  if [ -n "${1}" ]; then
    if [ -n "${logFile}" ]; then
      # Are using a log file
      echoStderr "[WARNING] $@" 2>&1 | tee --append $logFile
    else
      # Are NOT using a log file
      echoStderr "[WARNING] $@"
    fi
  else
    while read inputLine; do
      if [ -n "${logFile}" ]; then
        # Are using a log file
        echoStderr "[WARNING] ${inputLine}" 2>&1 | tee --append $logFile
      else
        # Are NOT using a log file
        echoStderr "[WARNING] ${inputLine}"
      fi
    done
  fi
}

score 1 · Answer 4 · answered Oct 15 '19 at 01:40

Thanks to people who posted their responses. I came up with my version which will add timestamp once per message.

#!/bin/bash
CURRENT_PID=$$
PROCESS_NAME=$(basename $0)

LOGFILE=/var/log/backup-monitor.log
function log_message {
  if [ -n "$1" ]; then
      MESSAGE="$1"
      echo -e "$(date -Iseconds)\t$PROCESS_NAME\t$CURRENT_PID\t$MESSAGE" | tee -a $LOGFILE
  else
      MESSAGE=$(tee)
      echo -e "$(date -Iseconds)\t$PROCESS_NAME\t$CURRENT_PID\t$MESSAGE" | tee -a $LOGFILE
  fi
}

log_message "Direct arguments are working!!"

echo "stdin also working" | log_message

chunky_pie · Answer 5 · 2022-06-13T22:48:44.960

You can create a function that optionally writes the STDOUT and STDERR from system commands to a log file or accepts arguments that will be written to a log file by doing the following:

_LOG='/some_dir/some_file'

function Log_Msg {

    #If no arguments are given to function....
    if [ -z "$@" ]; then
        
        #...then take STDOUT/STDERR as input and write to log file
        read && echo "$REPLY" | tee -a $_LOG \

    else
        #Take arguments that were given to function and write that to log file
        echo "$@" | tee -a $_LOG

    fi

}

#Logging from system commands example. The "|&" operator pipes STDOUT and STDERR to Log_Msg function
bad command |& Log_Msg

or

#Taking an argument as input and writing to log file
Log_Msg "Write this to log file"

Hope this helps!

score -1 · Answer 6 · answered Jan 01 '15 at 16:16

In my opinion, a timeout of 100ms ( -t 0.1 ) in read command will allow the LogMsg to handle input piping and parameters without waiting forever in case of no input.

function log(){ read -t 0.1 IN1
  echo $(date "+%Y/%m/%d %H:%M:%S")' ('$QMAKESPEC'): '$IN1 $* |tee -a $LogFile ;}
#test without, with pipe , with pipe and parameters , with parameters only
log ; echo foo | log ; echo foo | log bar ; log bar
2015/01/01 16:52:17 ():
2015/01/01 16:52:17 (): foo
2015/01/01 16:52:17 (): foo bar
2015/01/01 16:52:17 (): bar

tee -a duplicates to stdout and appends to $LogFile

have fun

score -2 · Answer 7 · answered Aug 10 '12 at 16:21

-2

There are 2 ways of doing so, first, which I think is better, is to create a bash file and pass the result to it like this:

make 2>&1 > ./LogMsg

the second way is to pass result as an argument to function:

LogMsg $(make 2>&1)

answered Aug 10 '12 at 16:21

PLuS

642
5
12

1

Your first option is unclear. Do you mean to pipe the output of make to `LogMsg` (which can't read from stdin as written)? Your second option will only process the first line from make, as `LogMsg` only processes its first argument. – chepner Aug 10 '12 at 16:54

Redirect stdout and stderr to Function

7 Answers7

Linked