1

I am not able to get to the div within so many nested divs. . . .

<div class="list-card-price">$2,600/mo</div>

. . .

I am using this method to retrive it:

from bs4 import BeautifulSoup
import requests
import lxml

response = requests.get(url="https://www.zillow.com/homes/for_rent/1-_beds/?.....")
data = response.text

soup = BeautifulSoup(data,"lxml")

price_list = []

price_tag = soup.select_one(name='div',class_="list-card-price")
print(price_tag)
mbithi
  • 15
  • 4
Johnny Dalal
  • 9
  • 3

2 Answers2

1

What happens?

  1. Always take a deeper look into your soup - There is the truth! It comes up with a captcha warning.

  2. Your selection will throw an error cause you mix syntax of find() and select_one()

How to fix?

  1. Take a closer look at this answer options to deal with captcha

  2. Change your selection to soup.select_one('div.list-card-price') to avoid the error TypeError: select_one() missing 1 required positional argument: 'selector'

HedgeHog
  • 22,146
  • 4
  • 14
  • 36
0

You can check the HTML that's being sent in the soup with "soup.text". When I ran the code, the website presented a captcha page to the requests module, not the actual webpage.

Finlay
  • 108
  • 10