How to grep the output of a command inside a shell script when scheduling using cron

Question

I have a simple shell script where I need to check if my EMR job is running or not and I am just printing a log but it does not seem to work properly when scheduling the script using cron as it always prints the if block statement because the value of "status_live" var is always empty so if anyone can suggest what is wrong here otherwise on manually running the script it works properly.

#!/bin/sh

status_live=$(yarn application -list | grep -i "Streaming App")

if [ -z $status_live ] 
then
  echo "Running spark streaming job again at: "$(date) &
else
  echo "Spark Streaming job is running, at: "$(date)
fi

Dudi Boy · Accepted Answer · 2021-12-29T17:49:18.847

Your script cannot run in cron because cron script has no environment context at all.

For example try to run your script as another use nobody that has no shell.

 sudo -u nobody <script-full-path>

It will fail because it has no environment context.

The solution is to add your user environment context to your script. Just add source to your .bash_profile

 sed -i "2a source $HOME/.bash_profile" <script-full-path>

Your script should look like:

#!/bin/sh
source /home/<your user name>/.bash_profile

status_live=$(yarn application -list | grep -i "Streaming App")

if [ -z $status_live ] 
then
  echo "Running spark streaming job again at: "$(date) &
else
  echo "Spark Streaming job is running, at: "$(date)
fi

Now try to run it again with user nobody, if it works than cron will work as well.

 sudo -u nobody <script-full-path>

Note that cron has no standard output. and you will need to redirect standard output from your script to a log file.

<script-full-path> >> <logfile-full-path>

SreehariGaddam · Answer 2 · 2021-12-29T06:17:31.977

0

# $? will have the last command status in bash shell scripting
# your complete command here below and status_live is 0  if it finds in grep (i.e. true in shell if condition.)
yarn application -list | grep -i "Streaming App"
status_live=$?
echo status_live: ${status_live}
if [ "$status_live" -eq 0 ]; then
    echo "success
else
    echo "fail" 
fi

edited Dec 29 '21 at 06:17

answered Dec 29 '21 at 06:09

SreehariGaddam

479
4
8

Actually the problem is that the above shell script was also running fine when I was manually running the script, but on scheduling it using a cron it does not work as expected and same is happening with this script as well. – Yash Tandon Dec 29 '21 at 09:07
[Why is testing “$?” to see if a command succeeded or not, an anti-pattern?](https://stackoverflow.com/questions/36313216/why-is-testing-to-see-if-a-command-succeeded-or-not-an-anti-pattern) – tripleee Dec 29 '21 at 11:22
@YashTandon looks like in cron, it might be running under different shell environment. – SreehariGaddam Dec 29 '21 at 15:42
@YashTandon SHELL=/bin/bash in the cron file before your command running, just to make sure its running under bash – SreehariGaddam Dec 29 '21 at 15:52
@tripleee $? will give always sucess or failure of previous command in shell – SreehariGaddam Dec 29 '21 at 15:53
It does, but you obviously need to read the linked question about why to generally avoid it. – tripleee Dec 29 '21 at 16:16
Probably `yarn` gives different output (because you haven't activated its environment, perhaps?) and then the syntax changes here will not change the result. – tripleee Dec 29 '21 at 16:17

How to grep the output of a command inside a shell script when scheduling using cron

2 Answers2

Linked