How to log output of multiple shell scripts

Question

I am not familiar with this platform, so if this is in the frozen section my apologies :P

I am working on upgrading a raspberry pi script project. It decodes NOAA APT Satellite images and it runs from AT Scheduler (I think) and scripts. Scripts are used to start recordings and do auto processing.

I have been having some problems and am trying to get a log of what is processed through the scripts, their are 3. I have tired adding something like ...) >> log.txt to the files but they are always empty.

I cant call them as sh -x script.sh >>log.txt because they are scheduled to trigger at different times and it would be a pain to replace all the calls.

Idealy i would like something i could add at the end of each script to log all the things they process and stick them in their own log file (script1.log, script2.log, script3.log)

Thanks!! Jake

edit: I was advised to post the scripts. These are not "mine" i got them off of an instructable and made some changes to fit my needs. And i would rather not screw them up more than i have. ideally i would like something i could put after the #!/bin/bash line where it would log all of the commands processed by the script.

Thanks!

Script 1, the main scheduling script. some of them have been comented out because i dont use NOAA 15 or Meteor M2.

#!/bin/bash
# Update Satellite Information

wget -qr https://www.celestrak.com/NORAD/elements/weather.txt -O /home/pi/weather/predict/weather.txt

#grep "NOAA 15" /home/pi/weather/predict/weather.txt -A 2 > /home/pi/weather/predict/weather.tle

grep "NOAA 18" /home/pi/weather/predict/weather.txt -A 2 >> /home/pi/weather/predict/weather.tle

grep "NOAA 19" /home/pi/weather/predict/weather.txt -A 2 >> /home/pi/weather/predict/weather.tle

#grep "METEOR-M 2" /home/pi/weather/predict/weather.txt -A 2 >> /home/pi/weather/predict/weather.tle

#Remove all AT jobs
for i in `atq | awk '{print $1}'`;do atrm $i;done

#Schedule Satellite Passes:
/home/pi/weather/predict/schedule_satellite.sh "NOAA 19" 137.1000

/home/pi/weather/predict/schedule_satellite.sh "NOAA 18" 137.9125

#/home/pi/weather/predict/schedule_satellite.sh "NOAA 15" 137.6200

script 2, the individual satellite scheduler. It uses information from the first script to find times the satellite is passing overhead.

#!/bin/bash

PREDICTION_START=`/usr/bin/predict -t /home/pi/weather/predict/weather.tle -p "${1}" | head -1`

PREDICTION_END=`/usr/bin/predict -t /home/pi/weather/predict/weather.tle -p "${1}" | tail -1`

var2=`echo $PREDICTION_END | cut -d " " -f 1`

MAXELEV=`/usr/bin/predict -t /home/pi/weather/predict/weather.tle -p "${1}" | awk -v max=0 '{if($5>max){max=$5}}END{print max}'`

while [ `date --date="TZ=\"UTC\" @${var2}" +%D` == `date +%D` ]; do

START_TIME=`echo $PREDICTION_START | cut -d " " -f 3-4`

var1=`echo $PREDICTION_START | cut -d " " -f 1`

var3=`echo $START_TIME | cut -d " " -f 2 | cut -d ":" -f 3`

TIMER=`expr $var2 - $var1 + $var3`

OUTDATE=`date --date="TZ=\"UTC\" $START_TIME" +%Y%m%d-%H%M%S`

if [ $MAXELEV -gt 28 ]

  then

    echo ${1//" "}${OUTDATE} $MAXELEV

    echo "/home/pi/weather/predict/receive_and_process_satellite.sh \"${1}\" $2 /home/pi/weather/${1//" "}${OUTDATE} /home/pi/weather/predict/weather.tle $var1 $TIMER" | at `date --date="TZ=\"UTC\" $START_TIME" +"%H:%M %D"`
fi

nextpredict=`expr $var2 + 60`

PREDICTION_START=`/usr/bin/predict -t /home/pi/weather/predict/weather.tle -p "${1}" $nextpredict | head -1`

PREDICTION_END=`/usr/bin/predict -t /home/pi/weather/predict/weather.tle -p "${1}"  $nextpredict | tail -1`

MAXELEV=`/usr/bin/predict -t /home/pi/weather/predict/weather.tle -p "${1}" $nextpredict | awk -v max=0 '{if($5>max){max=$5}}END{print max}'`

var2=`echo $PREDICTION_END | cut -d " " -f 1`

done

the final script takes care of recording the audio from the satellite at the specified frequency, calculated for dopler shift, auto decodes/processes it, and posts it to my archive and webserver.

#!/bin/bash

# $1 = Satellite Name

# $2 = Frequency

# $3 = FileName base

# $4 = TLE File

# $5 = EPOC start time

# $6 = Time to capture

sudo timeout $6 rtl_fm -f ${2}M -s 60k -g 45 -p 55 -E wav -E deemp -F 9 - | sox -t wav - $3.wav rate 11025
#pass start 150 was 90
PassStart=`expr $5 + 150`

if [ -e $3.wav ]

then

/usr/local/bin/wxmap -T "${1}" -H $4 -p 0 -l 0 -o $PassStart ${3}-map.png

/usr/local/bin/wxtoimg -m ${3}-map.png -e ZA $3.wav ${3}.png

/usr/local/bin/wxtoimg -m ${3}-map.png -e NO $3.wav ${3}.NO.png

/usr/local/bin/wxtoimg -m ${3}-map.png -e MSA $3.wav ${3}.MSA.png

/usr/local/bin/wxtoimg -m ${3}-map.png -e MCIR $3.wav ${3}.MCIR.png

/usr/local/bin/wxtoimg -m ${3}-map.png -e MSA-PRECIP $3.wav ${3}.MSA-PRECIP.png

/usr/local/bin/wxtoimg -m ${3}-map.png -e EC $3.wav ${3}.EC.png

/usr/local/bin/wxtoimg -m ${3}-map.png -e HVCT $3.wav ${3}.HVCT.png

/usr/local/bin/wxtoimg -m ${3}-map.png -e CC $3.wav ${3}.CC.png

/usr/local/bin/wxtoimg -m ${3}-map.png -e SEA $3.wav ${3}.SEA.png

fi


NOW=$(date +%m-%d-%Y_%H-%M)

mkdir /home/pi/weather/Pictures/${NOW}

sudo cp /home/pi/weather/*.png /home/pi/weather/Pictures/${NOW}/    #move pictures to date folder in pi/pictures

sudo mv /var/www/html/APT_Pictures/PREVIOUS/* /var/www/html/APT_Pictures/ARCHIVE      #move previous to archive

sudo mv /var/www/html/APT_Pictures/LATEST/* /var/www/html/APT_Pictures/PREVIOUS     #move latest pictures to previous folder

sudo cp /home/pi/weather/Pictures/${NOW} /var/www/html/APT_Pictures/LATEST -r     #copys date folder to latest

sudo cp /home/pi/weather/*-map.png home/pi/weather/Pictures/${NOW}/      #copys map to archive folder

##sudo mv /home/pi/weather/Pictures/${NOW}/*-map.png /home/pi/weather/maps     #moves map from /pi/pictures date to maps folder

sudo rm /home/pi/weather/*.png     #removes pictures from weather folder

sudo mv /home/pi/weather/*.wav /home/pi/weather/audio     #moves audio to  audio folder

Hi and welcome to StackOverflow! Would you mind editing your post and showing us what script.sh looks like? — Daniel Walker, May 18 '20 at 00:30

baltakatei · Answer 1 · 2020-05-19T19:39:51.000

0

Perhaps the scripts are outputing status messages to stderr instead of stdout (which your ...) >> log.txt method would have captured.)?

Here's how I'd capture stdout and stderr for debugging purposes.

$ /bin/bash script1.sh 1>>script1_stdout.log 2>>script1_stderr.log
$ /bin/bash script2.sh 1>>script2_stdout.log 2>>script2_stderr.log
$ /bin/bash script3.sh 1>>script3_stdout.log 2>>script3_stderr.log

Or combine the two streams into a single log file:

$ /bin/bash script1.sh 1>>script1.log 2>&1
$ /bin/bash script2.sh 1>>script2.log 2>&1
$ /bin/bash script3.sh 1>>script3.log 2>&1

The "1" in 1>> refers to stdout and the "2" in 2>> refers to stderr.

Edit: If you want to continue to see stdout/stderr messages and still write them to file, use tee as described here. tee prints stdin it receives, writes a copy of stdout to the file path provided.

$ /bin/bash script1.sh 2>&1 | tee script1.log
$ /bin/bash script2.sh 2>&1 | tee script2.log
$ /bin/bash script3.sh 2>&1 | tee script3.log

Reference about stdout and stderr.

edited May 19 '20 at 19:39

answered May 18 '20 at 00:53

baltakatei

113
5

I am looking for something i can include in the script. The way you have it written (/bin/bash script1.sh 1>>script1.log...) looks like calling it from the command line. Is there a way i can write this into the script withount including a call to itself/other scripts? – Jake May 18 '20 at 13:17
Huh Duh, I should have used my brain (really bad about doing that, it was early and i had just woken up). i just used ``` (code stuff )>>log.log 2>&1 ``` and now it works! However it stopped outputing errors to the terminal, which is kind of annoying but its ok. thanks for your help!!! – Jake May 18 '20 at 15:37
To preserve stdout/stderr messages, try `( code stuff ) 2>&1 | tee log.log`. See [this](https://stackoverflow.com/a/13591423). – baltakatei May 19 '20 at 19:42

How to log output of multiple shell scripts

1 Answers1