Showing Empty DataFrame as output

Question

This is my code. I tried to concatenate the data but it is showing the empty dataframe. I want the details of symbols provided in the list as output which will be in the form of dataframe.

import pandas as pd
from nsepy import get_history
from datetime import date


symbol=['SBIN','ITC']

data1=[]
data1= pd.DataFrame(data1)
counter=0
for x in symbol:
    data = get_history(symbol=x, start=date(2022,4,25), end=date(2022,5,5), index= True)
    data = pd.DataFrame(data)
    data1= pd.concat([data1,data])
    print(x)
    print(data1)

The output I am getting is:

SBIN
Empty DataFrame
Columns: [Open, High, Low, Close, Volume, Turnover]
Index: []
ITC
Empty DataFrame
Columns: [Open, High, Low, Close, Volume, Turnover]
Index: []

What is the intention behind the loop? Are you using it because you want other indices as well? Or do you want just `SBIN`? Your question needs to make things more clear. What is your intention? What do you want your final output to look like? It is unclear to me if you want consolidated dataframe that contains data from `NIFTY50`, `SBIN` etc etc. Because if that is the case, you have gone and put a single item in your list further confusing all of us because now I'm sure if you made a mistake with the list or if you did that intentionally with a future goal to add items to the list — Lihka_nonem, May 09 '22 at 07:41
@Timus, I can confirm that what you have said above indeed works. But its not a one stop solution. It doesn't work for `NIFTY`, `INDIAVIX`, `NIFTY NEXT 50` and countless more for sure. For the indices I just mentioned, it should be `index=True`. I am editing my answer, to account for ITC. — Lihka_nonem, May 10 '22 at 07:32
@Lihka_nonem Yes, indeed. (It's either a pretty messed up API or I don't understand it.) — Timus, May 10 '22 at 07:38

Lihka_nonem · Answer 1 · 2022-05-10T07:38:55.547

A couple of things to note here with this solution.

The quality of the data is not great. What I mean by this is that SBIN has x number of fields filled while NIFTY has y number of fields filled. So when you concat to create a consolidated dataframe, you get a dataframe with 'NAN' values for the x and y fields of NIFTY and SBIN respectively. This is a problem with the data and not something you can do much about.
The documentation mentions using data[['Close']].plot() to treat the NAN values. I have and from what I've observed in the data is that, it does remove, the NAN values but replaces it with an empty string.
Different indices have different ways of hitting the API which is a terrible design decision on their part. What I noticed from their documentation is that indices like NIFTY, INDIAVIX, NIFTY NEXT 50 all require the index=True parameter inside the get_history(). However if you set index=True and try to get the data for SBIN, you receive, an empty dataframe. Why they have done so is baffling but you are going to have live with the limitations of their API unless you are able to download this database.

With those points out of the way, you can use the following code.

list_indices = ['SBIN', 'INDIAVIX', 'ITC']

# Using a dictionary for O(1) look up time. 
indices_with_index_false = {'SBIN': '', 'ITC': ''}
consolidated_dataframe = pd.DataFrame()
for index in list_indices:
    if index in dict_index_false:
        data = get_history(symbol=index, start=date(2015, 1, 1), end=date(2015, 1, 31), index=False)
            data[['Close']].plot()
    else:
        data = get_history(symbol=index, start=date(2015, 1, 1), end=date(2015, 1, 31), index=True)
            data[['Close']].plot()

    consolidated_dataframe = pd.concat([consolidated_dataframe, data], sort=False)

consolidated_dataframe.to_csv("test.csv")

I recommend taking a look at this output in Excel or whatever is comfortable for you when viewing CSV files.

Showing Empty DataFrame as output

1 Answers1