1

Im new to bs4 and requests and I'm trying to webscrape Amazon for the price of some random product (in this case an eco dot 3). I dont understand why my web scraper always returns "None" when I run it. Clearly the element I am looking for is present, but still when I try and use the find function it doesnt work. I dont know if it helps but here is the tutorial I was following: https://www.youtube.com/watch?v=Bg9r_yLk7VY&t=594sThank you very much! Also here's my code

from bs4 import BeautifulSoup


URL = 'https://www.amazon.de/dp/B07NQCVHQ6/ref=gw_de_desk_h1_aucc_db_dbckpscs_qh_0520_v1_t1?pf_rd_r=WY9A7Y14N1T2NGG5KT9S&pf_rd_p=56ce1aab-22bc-4377-bb24-b13391bb0efd'
headers = {"User-Agent": 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36'}

page = requests.get(URL, headers=headers)

soup = BeautifulSoup(page.content, 'html.parser')

price = soup.find(id="priceblock_ourprice")
print(price)

I dont know if it helps but here is the tutorial I was following: https://www.youtube.com/watch?v=Bg9r_yLk7VY&t=594s

USERNAME GOES HERE
  • 692
  • 1
  • 15
  • 29
Daniel Rübenacker
  • 25
  • 5

1 Answers1

1

you need to make simple changes just change this:

soup = BeautifulSoup(page.content, 'html.parser')

to

soup = BeautifulSoup(page.content, 'lxml')

output:

<span class="a-size-medium a-color-price priceBlockBuyingPriceString" id="priceblock_ourprice">48,73 ?</span>

And:

print(price.text)

Will give you only the price

about the difference between parsers, you can read on official page - - https://www.crummy.com/software/BeautifulSoup/bs4/doc/#differences-between-parsers

or in this question, there is explanation - Beautiful Soup and Table Scraping - lxml vs html parser

Roman
  • 1,883
  • 2
  • 14
  • 26
  • While this code may solve the question, [including an explanation](//meta.stackexchange.com/q/114762) of how and why this solves the problem would really help to improve the quality of your post, and probably result in more up-votes. Remember that you are answering the question for readers in the future, not just the person asking now. Please [edit] your answer to add explanations and give an indication of what limitations and assumptions apply. –  Jul 14 '20 at 11:31