Cron Pipe zcat on Bash Script not working

Question

I created a script based on the tutorial on this page: https://johnveldboom.com/posts/goaccess-automated-reports-last-30-days-via-cron/. When the script is run from the terminal it works perfectly. The problem is when cron runs the script, pipe does not seem to work.

I Googled for possible solutions. I tried adding "-c" to the bash on cron but that did not help.

This is the Script

 #!/bin/bash
 # filename: goaccess.sh (with +x permission)
 HOST="myHost"
 GOACCESSREPORT_DIR=/home/user/goaccess_reports/
 DATE=$(date +'%Y.%m')
 /bin/zcat `find /var/log/nginx/ -name "myhost_access.log.*.gz" -mtime -20` | goaccess > $GOACCESSREPORT_DIR/$HOST-monthly-$DATE.html
 echo "My Host GoAccess Report" | sudo mail -s "My Host GoAccess Report" email@test.com -A $GOACCESSREPORT_DIR/$HOST-monthly-$DATE.html

This is my cron

 00 22 * * 5 /bin/bash -c /home/user/goaccess.sh

The output file information is below, which lets me to think that piping is not working:

GoAccess - 1.2 Usage: goaccess [filename] [ options ... ] [-c][-M][-H][-q][-d][...] The following options can also be supplied to the command: Log & Date Format Options --date-format= - Specify log date format. e.g., %d/%b/%Y --log-format= - Specify log format. Inner quotes need to be escaped, or use single quotes. --time-format= - Specify log time format. e.g., %H:%M:%S User Interface Options -c --config-dialog - Prompt log/date/time configuration window. -i --hl-header - Color highlight active panel. -m --with-mouse - Enable mouse support on main dashboard. --color= - Specify custom colors. See manpage for more details and options. --color-scheme=<1|2|3> - Schemes: 1 => Grey, 2 => Green, 3 => Monokai. --html-custom-css= - Specify a custom CSS file in the HTML report. --html-custom-js= - Specify a custom JS file in the HTML report. --html-prefs= - Set default HTML report preferences. --html-report-title=

Thank you in advance.

@JosephSible-ReinstateMonica, thank you for your response. I added the full file path and still does not work. Sorry for the late response. — Rayed, May 26 '20 at 19:51
Does it still work from the terminal with the fully qualified path? — Joseph Sible-Reinstate Monica, May 26 '20 at 20:28
As a test, what do you get if you replace `goaccess` with `echo goaccess`, so that it shows you the exact parameters it's passing? — Joseph Sible-Reinstate Monica, May 26 '20 at 20:52
Like this: /bin/zcat `find /var/log/nginx/ -name "myhost_access.log.*.gz" -mtime -20` | echo goaccess > $GOACCESSREPORT_DIR/$HOST-monthly-$DATE.html ??? — Rayed, May 26 '20 at 21:04
Yes. It obviously won't work, but the output it produces will be useful. Please post the result both from the terminal and from cron. — Joseph Sible-Reinstate Monica, May 26 '20 at 21:05
Yes for both. It is very strange. I will instead save the zcat out to a file and read the file with goaccess. Looks like piping does not want to play nice with cron. thanks for your help. — Rayed, May 26 '20 at 21:33

score 3 · Accepted Answer · answered Dec 14 '20 at 03:22

3

I had the same problem and found the answer here.

You need to use a dash after the goaccess command to tell it you are piping the logs in:

/bin/zcat `find /var/log/nginx/ -name "myhost_access.log.*.gz" -mtime -20` | goaccess - > $GOACCESSREPORT_DIR/$HOST-monthly-$DATE.html

answered Dec 14 '20 at 03:22

Hugh

371
1
8

1

Thank you for the help and source. one missing "-" Wow – Rayed Dec 30 '20 at 19:58

score 1 · Answer 2 · answered Jan 12 '23 at 13:28

This helped me also, thank you all.

I found that this worked best for me:

#!/bin/bash

# create the report
/usr/bin/zcat -f /var/log/nginx/nhh-access.* | /home/linuxbrew/.linuxbrew/bin/goaccess - --log-format=COMBINED > /var/www/logs/nhh.html
/usr/bin/zcat -f /var/log/nginx/no-host-access.* | /home/linuxbrew/.linuxbrew/bin/goaccess - --log-format=COMBINED > /var/www/logs/no-host.html

# change the title:
/usr/bin/sed -i 's/Server\&nbsp/nhh\&nbsp/' /var/www/logs/nhh.html
/usr/bin/sed -i 's/Server\&nbsp/no-host\&nbsp/' /var/www/logs/no-host.html

cron:

# every 13 minutes:
*/13 * * * * /var/www/logs/goaccess-job.sh > /var/www/logs/go.log 2>&1

Cron Pipe zcat on Bash Script not working

2 Answers2