1

This short script scrapes some log files daily to create a simple extract. It works from the command line and when I echo the $cmd and copy/paste, it also works. But it will breaks when I try to execute from the script itself.

I know this is a nightmare of patterns that I could probably improve, but am I missing something simple to just execute this correctly?

#!/bin/bash
priorday=$(date --date yesterday +"%Y-%m-%d")
outputfile="/home/CCHCS/da14/$priorday""_PROD_message_processing_times.txt"
cmd="grep 'Processed inbound' /home/rules/care/logs/RootLog* | cut -f5,6,12,16,18 -d\" \" | grep '^"$priorday"' | sed 's/\,/\./' | sed 's/ /\t/g' | sed -r 's/([0-9]+\-[0-9]+\-[0-9]+)\t/\1 /' | sed 's/  / /g' | sort >$outputfile"
printf "command to execute:\n"
echo $cmd
printf "\n"
$cmd

ouput:

./make_log_extract.sh command to execute: grep 'Processed inbound' /home/rules/care/logs/RootLog.log /home/rules/care/logs/RootLog.log.1 /home/rules/care/logs/RootLog.log.10 /home/rules/care/logs/RootLog.log.11 /home/rules/care/logs/RootLog.log.12 /home/rules/care/logs/RootLog.log.2 /home/rules/care/logs/RootLog.log.3 /home/rules/care/logs/RootLog.log.4 /home/rules/care/logs/RootLog.log.5 /home/rules/care/logs/RootLog.log.6 /home/rules/care/logs/RootLog.log.7 /home/rules/care/logs/RootLog.log.8 /home/rules/care/logs/RootLog.log.9 | cut -f5,6,12,16,18 -d" " | grep '^2014-01-30' | sed 's/\,/./' | sed 's/ /\t/g' | sed -r 's/([0-9]+-[0-9]+-[0-9]+)\t/\1 /' | sed 's/ / /g' | sort /home/CCHCS/da14/2014-01-30_PROD_message_processing_times.txt

grep: 5,6,12,16,18: No such file or directory

fedorqui
  • 275,237
  • 103
  • 548
  • 598
DaveA
  • 217
  • 3
  • 8
  • 1
    Basically, try not to do it: http://stackoverflow.com/questions/794728/variables-as-commands-in-bash-scripts – grebneke Jan 31 '14 at 16:05

1 Answers1

3

As grebneke comments, do not store the command and then execute it.

What you can do is to execute it but firstly print it: Bash: Print each command before executing?

priorday=$(date --date yesterday +"%Y-%m-%d")
outputfile="/home/CCHCS/da14/$priorday""_PROD_message_processing_times.txt"

set -o xtrace # <-- set printing mode "on"
grep 'Processed inbound' /home/rules/care/logs/RootLog* | cut -f5,6,12,16,18 -d\" \" | grep '^"$priorday"' | sed 's/\,/\./' | sed 's/ /\t/g' | sed -r 's/([0-9]+\-[0-9]+\-[0-9]+)\t/\1 /' | sed 's/  / /g' | sort >$outputfile"
set +o xtrace # <-- revert to normal
Community
  • 1
  • 1
fedorqui
  • 275,237
  • 103
  • 548
  • 598
  • +1, what is the difference between `set -x` and `set -o xtrace`, if any? – grebneke Jan 31 '14 at 16:11
  • From what I see in http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_02_03.html, it is exactly the same. Just short & long notation. – fedorqui Jan 31 '14 at 16:13
  • 1
    Single-character shell options can also be used when starting `bash`: `bash -x` is equivalent to starting `bash` then executing `set -x`. – chepner Jan 31 '14 at 16:24
  • Thanks; very helpful. I was able to bracket the command with print command and then work through each piped command to get the syntax right: #grep 'Processed inbound' /home/rules/care/logs/RootLog* | cut -f5,6,12,16,18 -d' ' | grep '^"$priorday"' | sed 's/\,/\./' | sed 's/ /\t/g' | sed -r 's/([0-9]+\-[0-9]+\-[0-9]+)\t/\1 /' | sed 's/ / /g' | sort >$outputfile grep 'Processed inbound' /home/rules/care/logs/RootLog* | cut -f5,6,12,16,18 -d' ' | grep "^$priorday" | sed 's/\,/\./' | sed 's/ /\t/g' | sed -r 's/([0-9]+\-[0-9]+\-[0-9]+)\t/\1 /' | sed 's/ / /g' | sort >$outputfile ` – DaveA Jan 31 '14 at 17:00