Scraping multiple data with same class name using BeautifulSoup

Question

I recently posted here, but I was not able to solve the problem so I'm asking again. I'm trying to scrape this website for 'Recent Sales' section (all the addresses): https://www.compass.com/agents/irene-vuong/

My code looks like this:

listings = []
for item in soup.findAll('a', {'class':'uc-listingCard-title'}):
    listings.append(item.get_text(strip=True))

print(listings)

My output is :

['256-258 Wyckoff Street', '1320 Glenwood Road', '1473 East 55th Street', '145 Winter Avenue', '25-02 Brookhaven Avenue']

However expected out would be:

['256-258 Wyckoff Street', '1320 Glenwood Road', '1473 East 55th Street', '145 Winter Avenue', '25-02 Brookhaven Avenue', '352 94th Street', '1754 West 12th Street', '2283 E 23rd st', '2063 Brown Street, '3423 Avenue U', '2256 Stuart Street']

Which contians all address as the class name is same as

<a class="uc-listingCard-title" href="`````" data-tn="listingCard-label-address"> adress here </a>

I don't understand why my code only gets first part but not the whole addresses when it has same class name.

Thank you in advance for any help.

++++ With suggestions:

for item in soup.findAll('div', attrs={'class': 'uc-listingCard-content'}):
    new = item.find('a', attrs={'class': 'uc-listingCard-title'})
    print(new.text)

I still only get the current listing addresses but NOT ALL addresses.

Can you share the relevant part of the HTML source code? Please provide a [mcve]. — AMC, Mar 04 '20 at 16:15
Does this answer your question? [Scraping data with multiple same class name using BeautifulSoup](https://stackoverflow.com/questions/60515908/scraping-data-with-multiple-same-class-name-using-beautifulsoup) — AMC, Mar 04 '20 at 21:09

yehezkel horoviz · Answer 1 · 2020-03-04T16:09:46.457

0

try:

from bs4 import BeautifulSoup

url = 'https://www.compass.com/agents/irene-vuong/'
url = requests.get(url)
tags = BeautifulSoup(url.text, 'html')
smaple_list=[]
for tag in tags.findAll('div', attrs={'class': 'uc-listingCard-content'}):
    new_tag = tag.find('a', attrs={'class': 'uc-listingCard-title'})
    print(new_tag.text)

edited Mar 04 '20 at 16:09

answered Mar 04 '20 at 15:25

yehezkel horoviz

198
3
10

Hi!!! It does get all the address, but I only want to scrap the text out, so I did .text but I get an attribute error of "ResultSet object has no attribute 'text'. You're probably treating a list of items like a single item. Did you call find_all() when you meant to call find()?". I tried "find" but I get different error, what do I need to do? Thank you so much in advance!! – Sarah Mar 04 '20 at 15:36
take a look: https://stackoverflow.com/questions/38053464/extracting-src-from-beautifulsoup-tag – yehezkel horoviz Mar 04 '20 at 15:38
So I still get the current listing, but not all sales listings. I want to get "Recent Sales" addresses, which comes after current listing... – Sarah Mar 04 '20 at 16:17
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/209010/discussion-between-sarah-kim-and-yehezkel-horoviz). – Sarah Mar 04 '20 at 16:23
Are you still there? – Sarah Mar 04 '20 at 18:54

Scraping multiple data with same class name using BeautifulSoup

1 Answers1