1

Hello all after hours of google searching hoping somebody here can provide direction.

Currently I have a .sh file which I run in putty to kick off a process. The current way I do this is:

sh /filepath/file.sh >> /filepath/file_date.log

While this works; I find it cumbersome to make sure I do the >> and logfile path correct for each of the .sh files I invoke.

The current contents of my .sh file are as such:

#!/bin/bash
set -x

frstdt=`date -d "last monday - 2 week" +"%m%d%Y"`;
scnddt=`date -d "last sunday - 1 week" +"%m%d%Y"`;
tday=`date -d "$date" +"%m_%d_%Y"`;
frstdtem=`date -d "last monday - 2 week" +"%m/%d/%Y"`;
scnddtem=`date -d "last sunday - 1 week" +"%m/%d/%Y"`

cd /filepath/
/filepath/cloak-hive -f query.hql

--Here I use hive -e to get some results from the query and write them to a .csv file.

ehco "Contents of email" | mailx -s "Subject" -a "Attchement" -me@email.com

What I am trying to understand is what I can add to the beginning of the .sh file to log all of the actions that take place, the map/reduce updates HIVE etc. I can see all this now when using the >> command when calling the .sh but am lookign to simplify this for others on the team. I have tried using exec trap exec; | tee and other version of that but can not get anything to work. I do not need to see anything in the console after I enter sh /filepath/filename.sh I just want everthing to show up in the log with the log file name having the dat it runs in the file name(which is why I have tday as a variable; I would like it to show as /filepath/filename_$tday.log

Sotark
  • 186
  • 11
  • `set -x` output goes to stderr and it is already there so redirecting stderr to stdout could be enough `sh /filepath/file.sh 2>&1 >> /filepath/file_date.log` – LMC Oct 05 '22 at 23:32
  • That is what I am currently doing. What I'm trying to understand is if there is a command I put in the .sh file that would eliminate the need for >>filepath/file_date.log. so that the log file is created automatically when running the .sh fiel – Sotark Oct 06 '22 at 00:17

3 Answers3

1

Running the script with sh is an error. See Difference between sh and bash

If I am guessing correctly what you are trying to ask, my suggestion would be to create a simple wrapper, like

#!/bin/sh
/filepath/file.sh >> /filepath/file_$(date +%F).log

Your scripts should have a valid shebang and you need to chmod +x both to be able to use that facility instead of spelling out sh or bash. Then you also don't need to keep track of which shell to use for which script.

I would also refactor your Bash script slightly.

#!/bin/bash
set -x

# Use modern POSIX syntax for command substitution
tday=$(date -d "$date" +"%m_%d_%Y")
frstdtem=$(date -d "last monday - 2 week" +"%m/%d/%Y")
scnddtem=$(date -d "last sunday - 1 week" +"%m/%d/%Y")
# Don't call the same date commands again; instead, just remove slashes
frstdt=${frstdtem//\//}
scnddt=$(scnddtem//\//}

# Probably avoid cd, but depends
# See https://stackoverflow.com/questions/45591428/what-exactly-is-current-working-directory
cd /filepath/
/filepath/cloak-hive -f query.hql

--Here I use hive -e to get some results from the query and write them to a .csv file.

# Don't misspell echo or Attachment
echo "Contents of email" | mailx -s "Subject" -a "Attachment" -me@example.com

Using the most insane possible date format will create troubles down the road; I would recommend using ISO 8601 machine-readable YYYY-MM-DD date format for everything except possibly human display (but better yet, train your humans) but I did not change that part.

tripleee
  • 175,061
  • 34
  • 275
  • 318
  • Thank you so much for the feedback. I am new to this and the processes I use are based on .sh files provided by the previous colleague who used them. I am doing my best to adapt and update where I can. For the wrapper portion is that a separate .sh file I save and call for each job, or is that a command that goes in the top of the .sh file under the #!/bin/bash. Sorry if I am misunderstanding your suggestion. – Sotark Oct 12 '22 at 14:02
  • 1
    That would be a new script which you save in a file and run instead of the main script, so you don't have to remember the redirections etc yourself; the script fills those in for you. But I obviously had to guess which parts to parametrize and whether this is automatable in the first place. If it always requires human input, make a script which accepts an argument (which turns up in `"$1"`) – tripleee Oct 12 '22 at 14:04
  • Thank you for that, that makes it clearer. I will try this out, and read up on the two links you provided. – Sotark Oct 12 '22 at 14:13
  • Also for the date command to remove slashes could you explain how that works. Because there are some files I have where the date is done with _ instead and I would like to remove those when necessary. – Sotark Oct 12 '22 at 14:34
  • 1
    Read about [parameter expansion](https://www.gnu.org/s/bash/manual/html_node/Shell-Parameter-Expansion.html) – tripleee Oct 12 '22 at 15:44
1

the best way to do it is:

command >> /filepath/logfile.log 2>&1

put it in your script,

thanks

0

I don't advise this praxis, but if you really want to do this, you can do a

exec >> /filepath/file_date.log 2>&1

as the first statement in your script.

user1934428
  • 19,864
  • 7
  • 42
  • 87