Add starttime and endtime to scrape yahoo fiance

Question

import requests
from bs4 import BeautifulSoup
from datetime import datetime

symbol = input("Symbol")
base_url = "https://finance.yahoo.com/quote/" + symbol + "/history/"

try:
    response = requests.get(base_url).text

except requests.exceptions.RequestException:
    print("Request gone wrong")

soup = BeautifulSoup(response, "html.parser")
table_data = soup.find_all("tr")

for i in table_data:
    print(i.text)

The finished product looks something like this:

DateOpenHighLowClose*Adj Close**Volume
Feb 25, 2021124.68126.46120.54120.99120.99148,199,540
Feb 24, 2021124.94125.56122.23125.35125.35110,691,500
Feb 23, 2021123.76126.71118.39125.86125.86157,859,100
Feb 22, 2021128.01129.72125.60126.00126.00103,607,600
Feb 19, 2021130.24130.71128.80129.87129.8787,525,900
Feb 18, 2021129.20130.00127.41129.71129.7196,648,100
Feb 17, 2021131.25132.22129.47130.84130.8497,918,500
Feb 16, 2021135.49136.01132.79133.19133.1980,206,200
Feb 12, 2021134.35135.53133.69135.37135.3760,029,300
Feb 11, 2021135.90136.39133.77135.13135.1364,154,400
Feb 10, 2021136.48136.99134.40135.39135.3973,046,600
Feb 09, 2021136.62137.88135.85136.01136.0176,774,200
Feb 08, 2021136.03136.96134.92136.91136.9171,297,200
Feb 05, 2021137.35137.42135.86136.76136.7675,524,000
Feb 05, 20210.205 Dividend
Feb 04, 2021136.30137.40134.59137.39137.1884,183,100
Feb 03, 2021135.76135.77133.61133.94133.7489,880,900
Feb 02, 2021135.73136.31134.61134.99134.7983,305,400
Feb 01, 2021133.75135.38130.93134.14133.94106,239,800
Jan 29, 2021135.83136.74130.21131.96131.76177,180,600
Jan 28, 2021139.52141.99136.70137.09136.89142,621,100
Jan 27, 2021143.43144.30140.41142.06141.85140,843,800
Jan 26, 2021143.60144.30141.37143.16142.9598,390,600
Jan 25, 2021143.07145.09136.54142.92142.71157,611,700
Jan 22, 2021136.28139.85135.02139.07138.86114,459,400
Jan 21, 2021133.80139.67133.59136.87136.67120,529,500
Jan 20, 2021128.66132.49128.55132.03131.83104,319,500
Jan 19, 2021127.78128.71126.94127.83127.6490,757,300
Jan 15, 2021128.78130.22127.00127.14126.95111,598,500
Jan 14, 2021130.80131.00128.76128.91128.7290,221,800
Jan 13, 2021128.76131.45128.49130.89130.6988,636,800
Jan 12, 2021128.50129.69126.86128.80128.6191,951,100
Jan 11, 2021129.19130.17128.50128.98128.79100,620,900
Jan 08, 2021132.43132.63130.23132.05131.85105,158,200
Jan 07, 2021128.36131.63127.86130.92130.72109,578,200
Jan 06, 2021127.72131.05126.38126.60126.41155,088,000
Jan 05, 2021128.89131.74128.43131.01130.8197,664,900
Jan 04, 2021133.52133.61126.76129.41129.22143,301,900
Dec 31, 2020134.08134.74131.72132.69132.4999,116,600

And so on

What I want to do is also put a Startdate and EndDate so I don't just scrape the first page of yahoo finace off. I have searched online, but it's all for JavaScript. Any input would be helpful. Let me know if you need any more details.

look at this URL https://finance.yahoo.com/quote/ONGC.NS/history?period1=1609459200&period2=1614297600&interval=1d&filter=history&frequency=1d&includeAdjustedClose=true ... — Deven Ramani, Feb 26 '21 at 05:17
@DevenRaman, i see this number 1609459200, i am guessing it is supposed to represent a date correct? What date format is this? — Wilfred, Feb 26 '21 at 05:19
for more detail on converting date to datenum you can refer [this link](https://stackoverflow.com/questions/13965740/converting-matlabs-datenum-format-to-python) — Deven Ramani, Feb 26 '21 at 05:25
import calendar, time; calendar.timegm(time.strptime('2021-01-01 00:00:00', '%Y-%m-%d %H:%M:%S')) #output = 1609459200 — Deven Ramani, Feb 26 '21 at 05:53

score 0 · Answer 1 · answered Feb 26 '21 at 05:56

It's epoch, so a date just needs to be converted

import datetime
print(int(datetime.datetime(2020,3,22).timestamp()))

1584853200

see here to test: https://www.epochconverter.com/

However, consider using one of several modules that do this for you - see this article https://medium.com/@jouneidraza522/yahoo-finance-api-to-get-stocks-tickers-data-in-python-c49820249a18

Deven Ramani · Answer 2 · 2021-02-26T06:07:00.987

Here is your modified full working code

import requests
from bs4 import BeautifulSoup
from datetime import datetime
import calendar, time

symbol = input("Symbol")
start_date = input("Start date in yyyy-dd-mm format")
end_date = input("end date in yyyy-dd-mm format")

start_datenum = calendar.timegm(time.strptime(start_date + ' 00:00:00', '%Y-%m-%d %H:%M:%S'))
end_datenum = calendar.timegm(time.strptime(end_date + ' 00:00:00', '%Y-%m-%d %H:%M:%S'))

base_url = "https://finance.yahoo.com/quote/" + symbol + "/history?period1=" + str(start_datenum) + "&period2=" + str(end_datenum) +"&interval=1d&filter=history&frequency=1d&includeAdjustedClose=true"


try:
    response = requests.get(base_url).text

except requests.exceptions.RequestException:
    print("Request gone wrong")

soup = BeautifulSoup(response, "html.parser")
table_data = soup.find_all("tr")

for i in table_data:
    print(i.text)

Add starttime and endtime to scrape yahoo fiance

2 Answers2