Bash script runs manually, but fails on crontab

Question

I'm a newbie to shell scripting. I have written a shell script to do incremental backup of MySQL database.The script is in executable format and runs successfully when executed manually but fails when executed through crontab.

Crontab entry is like this :

*/1 * * * * /home/db-backup/mysqlbackup.sh

Below is the shell script code -

#!/bin/sh
MyUSER="root"       # USERNAME
MyPASS="password"         # PASSWORD
MyHOST="localhost"  # Hostname
Password="" #Linux Password

MYSQL="$(which mysql)"
if [ -z "$MYSQL" ]; then
echo "Error: MYSQL not found"
exit 1
fi
MYSQLADMIN="$(which mysqladmin)"
if [ -z "$MYSQLADMIN" ]; then
    echo "Error: MYSQLADMIN not found"
    exit 1
fi
CHOWN="$(which chown)"
if [ -z "$CHOWN" ]; then
    echo "Error: CHOWN not found"
    exit 1
fi
CHMOD="$(which chmod)"
if [ -z "$CHMOD" ]; then
    echo "Error: CHMOD not found"
    exit 1
fi

GZIP="$(which gzip)"
if [ -z "$GZIP" ]; then
    echo "Error: GZIP not found"
    exit 1
fi
CP="$(which cp)"
if [ -z "$CP" ]; then
    echo "Error: CP not found"
    exit 1
fi
MV="$(which mv)"
if [ -z "$MV" ]; then
    echo "Error: MV not found"
    exit 1
fi
RM="$(which rm)"
if [ -z "$RM" ]; then
    echo "Error: RM not found"
    exit 1
fi
RSYNC="$(which rsync)"
if [ -z "$RSYNC" ]; then
    echo "Error: RSYNC not found"
    exit 1
fi

MYSQLBINLOG="$(which mysqlbinlog)"
if [ -z "$MYSQLBINLOG" ]; then
    echo "Error: MYSQLBINLOG not found"
    exit 1
fi
# Get data in dd-mm-yyyy format
NOW="$(date +"%d-%m-%Y-%T")"

DEST="/home/db-backup"
mkdir $DEST/Increment_backup.$NOW
LATEST=$DEST/Increment_backup.$NOW
$MYSQLADMIN -u$MyUSER -p$MyPASS flush-logs
newestlog=`ls -d /usr/local/mysql/data/mysql-bin.?????? | sed 's/^.*\.//' | sort -g | tail -n 1`
echo $newestlog
for file in `ls /usr/local/mysql/data/mysql-bin.??????`
do
        if [ "/usr/local/mysql/data/mysql-bin.$newestlog" != "$file" ]; then
     echo $file             
     $CP "$file" $LATEST         
        fi
done
for file1 in `ls $LATEST/mysql-bin.??????`
do
 $MYSQLBINLOG $file1>$file1.$NOW.sql 
 $GZIP -9 "$file1.$NOW.sql"     
 $RM "$file1"
done
$RSYNC -avz $LATEST /home/rsync-back

First of all, when scheduled on crontab it is not showing any errors. How can I get to know whether the script is running or not?
Secondly, what is the correct way to execute the shell script in a crontab. Some blogs suggest for change in environment variables. What would be the best solution

When I did $echo PATH, I got this

/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/mysql/bin:/opt/android-sdk-linux/tools:/opt/android-sdk-linux/platform-tools:~/usr/lib/jvm/jdk-6/bin

This is not to answer the question itself, but to suggest looking at 3rd-party tools for mysql backup scripts and save yourself some headaches. It's a very common thing to do, so there are lots of tools available. One that I've used is called automysqlbackup. I think that uses a dump approach rather than using the data files directly, so it may not suit you, but there are likely others that would. — jfrank, Jan 30 '13 at 23:23
By the way -- `MYSQL="$(which mysql)"` is not, generally speaking, a practice you should follow. The shell already caches PATH lookup results, and does so much more efficiently than by starting an external program such as `which`. — Charles Duffy, Feb 19 '17 at 02:17

amaurea · Accepted Answer · 2013-01-30T20:08:38.973

25

The problem is probably that your $PATH is different in the manual environment from that under which crontab runs. Hence, which can't find your executables. To fix this, first print your path in the manual environment (echo $PATH), and then manually set up PATH at the top of the script you run in crontab. Or just refer to the programs by their full path.

Edit: Add this near the top of your script, before all the which calls:

export PATH="/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/mysql/bin:/opt/android-sdk-linux/tools:/opt/android-sdk-linux/platform-tools:~/usr/lib/jvm/jdk-6/bin"

edited Jan 30 '13 at 20:08

answered Jan 30 '13 at 19:54

amaurea

4,950
26
35

1

It's probably better to execute the profile that sets up the environment in bash instead of copying it manually. There's a description of it here: http://www.linuxfromscratch.org/blfs/view/6.3/postlfs/profile.html – Jay Jan 30 '13 at 21:39
4

It's best to execute a cron job with as minimum an environment as possible, and specified exactly to meet the needs of the job. Executing your profile pulls in too much setup, irrelevant settings, and means you can accidentally break your cron job just by changing your interactive environment. For something like this mysql dump it should not be executing under *your* account anyway, it should be run under the mysql user crontab; you should not be able to login as the mysql user either, so PATH is unknown (system default) for that user. – Stephen P Feb 01 '13 at 21:21
@Jay - Stephen P is absolutely correct -- the best route is to use an explicit path, which is lightweight and direct, versus pulling in the whole profile, which is convoluted and excessive. In my opinion, your suggestion is squarely in the "bad practice" camp. – Chris Baker Jul 11 '13 at 20:19

score 14 · Answer 2 · answered Feb 01 '13 at 21:10

14

Another more generic way is to have cron run the user's bash logon process. In addition to the PATH, this will also pick up any LD_LIBRARY_PATH, LANG settings, other environment variables, etc. To do this, code your crontab entry like:

34  12  * * *   bash -l /home/db-backup/mysqlbackup.sh

answered Feb 01 '13 at 21:10

JimR

473
2
11

doesn't it trigger the same problem as mentioned before ? That all .bash_profile and .bash_rc will get run even if not needed ? I'm asking cause I'm facing the same problem, I like using BASH_ENV pointing to a minimal env script but that's not working when a shell is run through crontab. – R. Du Apr 12 '19 at 12:28

score 7 · Answer 3 · answered Jul 29 '16 at 03:56

My Issue was that I set the cron job in /etc/cron.d (Centos 7). It seems that when doing so I need to specify the user who executes the script, unlike when a cronjob is entered at a user level.

All I had to do was

*/1 * * * * root perl /path/to/my/script.sh
*/5 * * * * root php /path/to/my/script.php

Where "root" states that I am running the script as root. Also need to make sure the following are defined at the top of the file. Your paths might be different. If you are not sure try the command "which perl", "which php".

SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin

score 0 · Answer 4 · edited Aug 04 '18 at 18:07

0

Just import your user profile in the beginning of the script.

i.e.:

. /home/user/.profile

edited Aug 04 '18 at 18:07

MD004

581
1
7
19

answered Jul 16 '13 at 15:25

Enio Marques Junior

36
5

1

This isn't enough in my case – djjeck Feb 14 '15 at 04:15
Not a good idea. This creates a dependency to everything that .profile does. And as soon as .profile outputs just one character, you get a mail from cron. – Jens Aug 04 '18 at 18:35

score 0 · Answer 5 · edited Jun 29 '20 at 21:44

0

In your crontab file your script entry can have
```
* * * * * /bin/bash /path/to/your/script
```
Make sure to use full paths in your script. I have had a look through it and I have not seen any but just in case I missed it.

edited Jun 29 '20 at 21:44

β.εηοιτ.βε

33,893
13
69
83

answered Jun 29 '20 at 14:33

O. Kasule

21
1
1

score 0 · Answer 6 · answered Feb 14 '23 at 12:07

first run command env > env.tmp then run cat env.tmp copy PATH=.................. Full line and paste into crontab -e, line before your cronjobs. as like below

++=============================================================++
(# Edit this file to introduce tasks to be run by cron.
#
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
# and what command to run for the task
#
# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').
#
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
#
# Output of the crontab jobs (including errors) is sent through
# email to the user the crontab file belongs to (unless redirected).
#
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
#
# For more information see the manual pages of crontab(5) and cron(8)
PATH=/root/.nvm/versions/node/v18.12.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/sna>

# m h  dom mon dow   command
0 3 * * * bash /home/ubuntu/test.sh
* * * * * bash /home/ubuntu/test2.sh >> /home/ubuntu/cronlog.log
* 3 * * * bash /home/ubuntu/logscron.sh)
++=====================================================================++

Bash script runs manually, but fails on crontab

6 Answers6

Linked