MAC | Python Script Run Well Inserting Into Mysql table at Terminal but Not Working Running as Cron Job | MAC

Question

glad that someone can help.

The python script (a.py) run well inserting into Mysql (mydatabase/table1) from Terminal but mysql (mydatabase/table1) not able to be inserted or updated when run as cron job.

a.py parse all *.htm files in documents directory and the result lst_url insert into mydatabase/table1 field lst_url.

a.py located in documents directory.

The cron job is * * * * * documents/a.py

Thanks in advance.

Toh

a.py
-------------------------------------------------
import time
from bs4 import BeautifulSoup
from datetime import datetime
import pymysql
import glob
import os

conn = pymysql.connect(host='127.0.0.1', port=3306, user='root',passwd='XXXXXXXX',db='mysql', charset='utf8')
cur=conn.cursor()
cur.execute("USE mydatabase")

for page in glob.glob('documents/*.htm'):

    with open(page) as html_file:
        soup = BeautifulSoup(html_file, 'lxml')

    try:
        with conn.cursor() as cursor:
            for item in soup.find_all('div',class_='listing_info'):
                lst_url= item.find('a', href=re.compile(r'[/]([a-z]|[A-Z])\w+')).attrs['href'

                sql="INSERT INTO mydatabase.table1(lst_url) VALUES (%s)"
                cur.execute(sql,(lst_url))

        conn.commit()
    finally:
        #conn.close()
        print("Done")

You should put in some logging to see what the error is when running via the cronjob. — Spinor8, Aug 11 '18 at 13:42
If your on Linux or Unix add a Shebang line https://en.wikipedia.org/wiki/Shebang_(Unix) (#!/usr/bin/env python) or call it with * * * * * python documents/a.py — Dan-Dev, Aug 11 '18 at 13:49
Thank you Spinor8 and Dan Dev... Will try along the solution suggested... Shebang. Toh — Toh, Aug 12 '18 at 10:02

score 1 · Answer 1 · answered Aug 11 '18 at 14:50

Cron could be running it in a different python version / environment. I'd do both of what Spinor8 and Dan-Dev said, and I'll explain why.

As Spinor8 said you could log the python verison at the head of the program. Here's a great link explaining how.

As Dan-Dev said you should define the path to the python that you want to use with a shebang.

MAC | Python Script Run Well Inserting Into Mysql table at Terminal but Not Working Running as Cron Job | MAC

1 Answers1