Downloading Shares Gives "JSONDecodeError: Expecting value: line 1 column 1 (char 0)"

Question

I am downloading shared from Finance Yahoo. There is this error prompts:

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I have checked all the similar questions and applied but all in vain, that's why asking this question again specifying my problem.

I am downloading shares details of the top 100 current stocks of Nasdaq using this repo, that is basically for the prediction of stock shares based on financial analysis. there is this stage of downloading shares and saving them as Pandas DataFrame. The code is:

shares = []
tickers_done = []
for ticker in tqdm(tickers):
    if ticker in tickers_done:
        continue
    d = requests.get(f"https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/{ticker}?symbol={ticker}&padTimeSeries=true&type=annualPreferredSharesNumber,annualOrdinarySharesNumber&merge=false&period1=0&period2=2013490868")
    if not d.ok:
      time.sleep(300)
      d = requests.get(f"https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/{ticker}?symbol={ticker}&padTimeSeries=true&type=annualPreferredSharesNumber,annualOrdinarySharesNumber&merge=false&period1=0&period2=2013490868")
    ctn = d.json()['timeseries']['result']
    dct = dict()
    for n in ctn:
        type = n['meta']['type'][0]
        dct[type] = dict()
        if type in n:
            for o in n[type]:
                if o is not None:
                    dct[type][o['asOfDate']] = o['reportedValue']['raw']
    df = pd.DataFrame.from_dict(dct)
    df['symbol'] = ticker
    shares.append(df)
    tickers_done.append(ticker)
    time.sleep(1)
# save dataframe
df = pd.concat(shares)
df['date'] = df.index
df.to_csv(f"data/{project}/shares.csv", index=False)

And the error screen shot is:

And I have checked each ticker requests status_code as:

for x in tickers:

d = requests.get(f"https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/{x}?symbol={x}&padTimeSeries=true&type=annualPreferredSharesNumber,annualOrdinarySharesNumber&merge=false&period1=0&period2=2013490868")

print(d.status_code)

The results are all 403. Your kind help will be highly appreciable. Thanks!

But searching the link https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/AAPL?symbol=AAPL&padTimeSeries=true&type=annualPreferredSharesNumber,annualOrdinarySharesNumber&merge=false&period1=0&period2=2013490868in chrome by putting one of the tickers, like, AAPL, it gives some data, like,

Maybe the API rejects `requests` User Agent. Have you tried giving custom headers to the request? — OneCricketeer, Nov 27 '22 at 20:04
Also, have you tried using yfinance module instead of requests? — OneCricketeer, Nov 27 '22 at 20:05
@OneCricketeer, no dear. How do I use it here, could you please specify? — NeuralAI, Nov 27 '22 at 20:10
That confuses me here if API rejects it then why does pasting it in Chrome gives output, @OneCricketeer? Please have a look at the bottom of the question I have pasted there. — NeuralAI, Nov 27 '22 at 20:12
Does this answer your question? [JSONDecodeError: Expecting value: line 1 column 1 (char 0)](https://stackoverflow.com/questions/16573332/jsondecodeerror-expecting-value-line-1-column-1-char-0) — Ulrich Eckhardt, Nov 27 '22 at 20:16
@UlrichEckhardt, applied some of the answers, some threw out: _HTTPError: 403 Client Error: Forbidden for url_ and others like -FileNotFoundError: [Errno 2] No such file or directory:- — NeuralAI, Nov 27 '22 at 20:35
API can reject bots. Your web browser is not a Python script. It has a known User Agent from a Desktop machine. Like I said, try adding User Agent HTTP header **or** use yfinance, which should give you a similar dataframe — OneCricketeer, Nov 28 '22 at 19:26

score 0 · Answer 1 · answered Nov 26 '22 at 21:20

The error

Per your error trace, the below line is throwing an error:

ctn = d.json()['timeseries']['result']

The error is trying to tell you that the data in d is not JSON-formatted:

JSONDecodeError: Expecting value: line 1 column 1 (char 0) So basically the very first character (line 1, column 1) is not the expected { that starts JSON.

Add a print/breakpoint prior to the line above and see for yourself what data is in d (and whether or not it contains valid JSON).

The cause

As you point out towards the end:

The results are all 403. Your kind help will be highly appreciable. Thanks!

Since the requests.get calls are being rejected by the server, the responses don't contain valid JSON. Find out why the server is rejecting your calls (403 suggests you don't have access to the requested resource); start with the most basic call that returns [any] data and then add bits until you either get a working query, or an error.

Thanks, @Adam Smooch for the response. I tried to print d above the error-pointed line, it gives ****. The issue is with the link, I guess. But searching the link in chrome by putting one of the tickers, like, AAPL, it gives some data. If it were any access issues to the site then it should have given the same response there as well. I have attached a screen shot of the output from chrome to the question, please check it. — NeuralAI, Nov 27 '22 at 07:25

score 0 · Answer 2 · answered Apr 07 '23 at 07:30

I think this issue was solved in here: https://github.com/alvarobartt/investpy/issues/615

The problem is that investing.com has updated their website protection and that why the queries are getting rejeceted. However you can add the code in the example to the investpy package.

Downloading Shares Gives "JSONDecodeError: Expecting value: line 1 column 1 (char 0)"

2 Answers2

The error

The cause