This code used to work for me.
from bs4 import BeautifulSoup
from urllib.request import urlopen
search = 'some_website'
BeautifulSoup(urlopen(search), "lxml")
But now I get the following error.
HTTPError: HTTP Error 403: Forbidden
I can't do a simple request, because I need to scrape javascript information.
hdr = {'User-Agent': 'Mozilla/5.0'}
req = Request(site,headers=hdr)
page = urlopen(req)
soup = BeautifulSoup(page)
print(soup)
I get the following in the soup.
<noscript>Please enable JavaScript to view the page content.</noscript>
How do I get the JavaScript off a web page when I am am getting the "HTTP Error 403: Forbidden" error? Thanks for the help in advance.
I'm using python 3. Please let me know if you need more information.
