21

I have seen other questions that are similar but I can't find any real information on how to figure out the proper way to run a Bash script via Crontab. The .sh file is located in the user directory (in my case serverpilot).

The script is trying to copy the contents of the apps folder and send to my S3 bucket. The script works perfectly when run with sh backupS3.sh from the terminal but no joy via Cron. I've seen people reference PATH variables and the like but no idea where to even begin!

/backupS3.sh

#!/bin/sh
echo 'Started backing to S3'
date +'%a %b %e %H:%M:%S %Z %Y'
aws s3 sync /apps s3://bucketname
date +'%a %b %e %H:%M:%S %Z %Y'
echo 'Finished backing to S3'

crontab

*/10 * * * * /backupS3.sh >> backupS3.log

Can anyone point me at anything obvious I'm doing wrong? Thanks!

EDIT: I've added 2>&1 on to the end of the cron command and now I am getting something in the log file:

/bin/sh: 1: /backupS3.sh: not found

André Goldstein
  • 367
  • 1
  • 3
  • 9

3 Answers3

17

In pathnames, the leading / doesn't mean "home directory"; it always refers to the root of the file system, no matter who you're logged in as. You need to use the actual full, absolute path to the script (e.g. /home/serverpilot/backupS3.sh). If the crontab belongs to the same user whose home directory holds the script, you can use "$HOME"/backupS3.sh and the system will fill in their home directory path for you. Or, even more simply, just use ./backupS3.sh, since cron jobs start with their working directory equal to their owner's home directory.

If that doesn't work, then either what you mean by "user directory" is not the same as the POSIX concept of "home directory", or the script is not executable (which you can fix by running the command chmod +x backupS3.sh once).

If you're not sure what the full path is, just run the pwd command while you're in the same directory that holds the script, and put the output, followed by a slash, in front of the script name.

Mark Reed
  • 91,912
  • 16
  • 138
  • 175
  • 1
    Thanks, where would I run that `chmod` command? As a one-off in the terminal? – André Goldstein Jun 01 '16 at 17:17
  • 1
    Yes, it just has to be run once. – Mark Reed Jun 01 '16 at 17:18
  • Many thanks. Incidentally the `aws s3 sync /apps s3://bucketname` line within the script itself, will that know where to look or will that need to be the full server path also? Thanks again. – André Goldstein Jun 01 '16 at 17:21
  • If the directory is `/apps`, great. If it's `apps` under the user's home directory, then that also needs to be `"$HOME"/apps`. – Mark Reed Jun 01 '16 at 17:21
  • When Cron runs a job, it starts in your home driectory. Maybe all you need is `./backupS3.sh` – tripleee Jun 01 '16 at 17:22
  • Many thanks @MarkReed, I'm now receiving the following in my log `/srv/users/serverpilot/backupS3.sh: 4: /srv/users/serverpilot/backupS3.sh: aws: not found` Not sure if this is still saying it can't find the script or it's a problem with using the AWS CLI? – André Goldstein Jun 01 '16 at 17:26
  • The `aws` command is not in the cron job's `$PATH`. You'll need to specify the full path to it as well (which you can get with `which aws`). – Mark Reed Jun 01 '16 at 17:28
  • Thanks @tripleee I think I did try that initially. – André Goldstein Jun 01 '16 at 17:29
  • Obviously `/srv/serverpilot` is not the invoking user's home directory; but this answer sort of assumes that the script exists in your home directory, in which case the answer could be simpler. – tripleee Jun 01 '16 at 17:32
  • @tripleee I suspect it's more that the attempt with the `./` path was before the `chmod`. – Mark Reed Jun 01 '16 at 17:37
  • This answer is very helpful and has helped me get the script running. I'm now running into an AWS credentials error but I think that's due to configuring AWS originally as the root user – André Goldstein Jun 01 '16 at 17:49
  • Now working perfectly. Thanks all for your superb help. – André Goldstein Jun 01 '16 at 17:53
11

I've had the same problem where log file was created empty and the only way to see what is going on was to add 2>&1 at the end of the cron job.

*/10 * * * * /home/users/me/backupS3.sh >> /home/users/me/backupS3.log 2>&1

If anybody wonders what 2>&1 actually mean here is a short explanation:

2 - refers to a file descriptor used to identify stderr

1 - refers to a file descriptor used to identify stdout

So 2>&1 basically translates to: Redirect stderrtostdout

Community
  • 1
  • 1
foobar
  • 695
  • 9
  • 17
  • You could skip the `2>&1` part by using `&>> /home/users/me/backupS3.log` where `&>>` would redirect all outbound file descriptors to the log file, not just 1/`stdout`. – jlucktay Apr 15 '23 at 11:21
8

First, change #!/bin/sh to #!/bin/bash, next use the full path to the script (also .sh isn't needed in the file name)

*/10 * * * * /home/users/me/backupS3.sh >> /home/users/me/backupS3.log
SaintHax
  • 1,875
  • 11
  • 16
  • Thank you. When you say use the fullpath, I'm not sure where to point that to? I've added the Crontab to the `serverpilot` user (I also have access to the root of the server but thought it was preferable to login as serverpilot). I know the full server path to it, should I use that? – André Goldstein Jun 01 '16 at 17:16
  • 1
    The `bash` hint is a red herring; your script currently does not contain any Bash-specific constructs. – tripleee Jun 01 '16 at 17:34
  • Thanks for the info @triplee – André Goldstein Jun 01 '16 at 17:47
  • @tripleee red herring? It's additional and constructive since the user is posting this as a "bash question" No need for a down vote, as the path was the issue. – SaintHax Aug 09 '17 at 17:19
  • The advice to change the shebang is incorrect; it simply does not help solve the problem. – tripleee Aug 09 '17 at 17:52
  • @triplee then by your logic, after you give the advice (use full path), any additional help is incorrect. I'm afraid this goes against every principal of helping. Perhaps the problem is you stopped reading the answer when you got to the part you didn't like. – SaintHax Aug 11 '17 at 17:27
  • 1
    In fact, I read the entire answer, and it would be fine if you removed the part which is incorrect. There are many situations where that advice *would* help, but this isn't one of them. Recommending it anyway looks like [cargo cult programming](https://en.wikipedia.org/wiki/Cargo_cult_programming) to me, which I do think is based on poor principles (incidentally, not *"principals").* – tripleee Aug 12 '17 at 13:31