how to download and iterate over csv file

Question

I'm trying to download and iterate over csv file but I'm only reading the headers but no more lines after it tried using this answer but with no luck this is my code:

from datetime import datetime
import requests
import csv


def main():
    print("python main function")
    datetime_object = datetime.now().date()
    url = f'https://markets.cboe.com/us/equities/market_statistics/volume_reports/day/{datetime_object}/csv/?mkt=bzx'
    print(url)
    response = requests.get(url, stream=True)
    csv_content = response.content.decode('utf-8')
    print(csv_content)
    cr = csv.reader(csv_content.splitlines(), delimiter='~')
    my_list = list(cr)
    for row in my_list:
        print(row)


if __name__ == '__main__':
    main()

`yfinance` can get equity volumes. Do you have to download a CSV? — OneCricketeer, Dec 13 '20 at 15:23
@OneCricketeer I want to insert all the data from the csv file to mt db — yaodav, Dec 13 '20 at 18:53
I understand that. You still get a dataframe with that library that you can iterate over — OneCricketeer, Dec 14 '20 at 14:46
In any case, what happens if you just use `for line in csv_content.splitlines()` and skip using a csv.reader ? — OneCricketeer, Dec 14 '20 at 14:49

score 0 · Answer 1 · edited Dec 14 '20 at 15:10

0

cr = csv.reader(csv_content.splitlines(), delimiter='~')

change to

cr = csv.reader(csv_content.splitlines(), delimiter=',')

And check if You download full file or file with header only use URL in browser ;)

edited Dec 14 '20 at 15:10

Wondercricket

7,651
2
39
58

answered Dec 14 '20 at 14:58

PawelU

11
2

how to download and iterate over csv file

1 Answers1