Getting "406 Not Acceptable" when trying to scrape web using Python

Question

I am new to Python and web scraping. I am trying to scrape articles from a website but getting "406 Not Acceptable" when I run this code. Please help.

from requests import get
source_site_home_page = "https://www.readlightnovel.org/"
r = get(source_site_home_page)
print(r.text)

You are maybe sending too many requests too quickly. Does happen for the first request you make? — Henry Le Berre, May 05 '20 at 09:21
Does this help ? https://stackoverflow.com/questions/56101612/python-requests-http-response-406 — Henry Le Berre, May 05 '20 at 09:23
@HenryLeBerre I got the same error when I sent a request for the first time. — Faheem Akhtar, May 06 '20 at 07:28
@HenryLeBerre I have tried by adding user agent and it's throwing response code 200. No output. I am sorry if this is a dumb question/explanation. I have just watched a video on python scraping and started coding. — Faheem Akhtar, May 06 '20 at 07:33
@VishalUpadhyay What could be the reason? Why I'm not getting the output. — Faheem Akhtar, May 06 '20 at 07:33

Henry Le Berre · Accepted Answer · 2020-05-06T07:50:47.800

1

Add the user agent flags like in this thread : Python Requests HTTP Response 406

r.status_codegives you code 200 because the request was successful!

The contents of the page should be in r.text

edited May 06 '20 at 07:50

answered May 06 '20 at 07:44

Henry Le Berre

910
9
18

Getting "406 Not Acceptable" when trying to scrape web using Python

1 Answers1