Write to file in a schedule task Python

Question

I have a python script that crawls the web every 10secs (scheduling task), and i need that data to be saved in a file format. Problem is that I am only able to save the last set of data. I guess the other data is overwritten by the schedule task.

import sched
import time
from bs4 import BeautifulSoup
import requests
import datetime

scheduler = sched.scheduler(time.time, time.sleep)
url = 'https://in.finance.yahoo.com/q?s=AAPL'

def execute_async_task(address):
    requested = requests.get(address)
    data = requested.text
    soup = BeautifulSoup(data, 'html.parser')
    for link in soup.findAll('span', {'id': 'yfs_l84_aapl'})[0]:
        if link:
            f = open('PlotData.txt', 'w')
            f.write("stock_price:"+str(link)+"\n")
            time.sleep(0.05)
            scheduler.enter(10, 1, execute_async_task, (url,))


scheduler.enter(0, 1, execute_async_task, (url,))
scheduler.run()

I am relatively new, to python.

Won't this create too many files? I just want a effective method to write to same file! — Karan Kumar, Jun 01 '15 at 13:22
`open('PlotData.txt','a')` will open the file in append mode which will not overwrite the file as in `w` mode you can rotate the file after the file has reached a certain size — cmidi, Jun 01 '15 at 13:24

score 3 · Accepted Answer · answered Jun 01 '15 at 13:24

3

Use f = open('PlotData.txt', 'a') instead of f = open('PlotData.txt', 'w')

'w': Overwrite the existing file

'a': Append to the existing data in the file

answered Jun 01 '15 at 13:24

Yahya Yahyaoui

2,833
23
30

That worked! thanks! – Karan Kumar Jun 01 '15 at 13:26
Not at all :-) Feel free to validate my answer :-) – Yahya Yahyaoui Jun 01 '15 at 13:26

Write to file in a schedule task Python

1 Answers1