how to schedule a python code to run daily?

Question

def job():
# Define variables 
  tickers = "AAPL GOOGL"
  start = "2021-08-01"
  end = date.today()
  tickers_split = tickers.split()
  df = pd.DataFrame()
  tickers_split
  for ticker in tickers_split:
      data = pdr.get_data_yahoo(ticker, start=start, end=end, interval='d')
      df[ticker] = data['Adj Close']
  print(df.shape)
  display(df)
  df.to_csv(r'path.csv')

schedule.every().day.at("11:10").do(job)
while True:
   schedule.run_pending()
   time.sleep(1)

I have the above code to pull stock data and then save it on my local path and would like to make it run daily at specific time automatically. I am using schedule but it dsnt do what it is supposed to do. Do I need to have the code open for it to run? If thats the case, what the options are to execute python codes wo having them open or have my computer keep awake all the time?

score 2 · Answer 1 · answered Sep 27 '21 at 19:18

The prescribed way is to put the python code in AWS Lambda and then have an AWS Cloudwatch event with a cron expression invoke the AWS Lambda on a schedule and then save the results in S3 or some data store.

Above is the cheapest option both operationally and from a cost perspective as you are only paying for it when the AWS Lambda executes.

score 1 · Answer 2 · answered Sep 27 '21 at 18:08

1

Your OS probably provides a job scheduler (an application that runs in the background and can perform these scheduling tasks for you) for this purpose.

The typical tools are cron for Mac and Linux, and windows task scheduler for Windows. You can also look into other ones, or easier-to-use wrappers that may be available.

Important to note that a job scheduler can only run when your computer is turned on. If you need a thing that can run 24/7, you might look into a cloud provider or the like, which usually provide scheduling services of their own.

answered Sep 27 '21 at 18:08

Kaia

862
5
21

I am trying to avoid task scheduler and as you said cron is not for my system. I hoped schedule would work. I want to know why it dsnt do what it is supposed to do. – baharak Al Sep 27 '21 at 18:13
1

To launch a program, you have to have a program that determines whether it's time to launch it, and executes the code to launch it. That means either you need a program you wrote running in the background, or an existing program running in the background. The other option is to [launch the script on startup](https://stackoverflow.com/q/4438020/1275942) and keep the script running in the background. [See here](https://stackoverflow.com/q/2725754/1275942) for a number of solutions. – Kaia Sep 27 '21 at 18:26
1

Okay, I decided to use task scheduler and it dsnt not work. I have been looking at every explanation online and still the schedule dsnt launch at the specified time. I am thinking is it because I am importing packages in my script? almost all the examples I found have been simple scripts w/o import. If thats the case, then what the options are for the codes that have package install? – baharak Al Sep 28 '21 at 18:36
Packages should work. It might be helpful to run a small script to isolate problems; if the small script is working, it means there's a bug in your code. If the small script isn't working, it means there's an issue with how you're configuring task scheduler. – Kaia Sep 30 '21 at 19:13

score -1 · Answer 3 · answered Sep 27 '21 at 18:00

This video helped me with the same problem:

https://www.youtube.com/watch?v=aqnJvXOIr6g&ab_channel=KeithGalli

As I didn't want to keep my Laptop running I used the AWS setup. But I think this could also work with other hosting options where you can install your sheduler.

how to schedule a python code to run daily?

3 Answers3