0

I am trying to find the children div for a specific div on a website using beautifulSoup.

I have inspired myself from this answer : Beautiful Soup find children for particular div

However, when I want to retrieve all the content of the divs with class='row' which has a parent div with class="container search-results-wrapper endless_page_template" as seen below:enter image description here My problem is that it only retrieves the content of the first div class='row'.

I am using the following code :

    boatContainer= page_soup.find_all('div', class_='container search-results-wrapper endless_page_template')
    for row in boatContainer:
        all_boats = row.find_all('div', class_='row')
        for boat in all_boats:
           print(boat.text)

I apply this on this website. What can I do so that my solution retrieves the data of the divs from class='row' which belong in the div class='container search-results-wrapper endless_page_template' ?

Community
  • 1
  • 1
colla
  • 717
  • 1
  • 10
  • 22
  • Does this answer your question? [How to find children of nodes using BeautifulSoup](https://stackoverflow.com/questions/6287529/how-to-find-children-of-nodes-using-beautifulsoup) – sushanth Jun 02 '20 at 12:03
  • Unfortunately no, it does the same thing on the website I want to use. It only shows the text content for the first div row – colla Jun 02 '20 at 12:09
  • Works exactly as expected on my machine, are you sure your initial soup is correct ? – Samuel Middendorp Jun 02 '20 at 12:14
  • I edited my question so you can have a look – colla Jun 02 '20 at 12:18
  • @colla can you tell me what data are you particularly interested in scraping from this website.. Needless to state, I'd read your question several times, yet couldn't make heads or tails out off it. – mnm Jun 02 '20 at 12:24

1 Answers1

1

Use response.content instead of response.text.

you're also not requesting the correct url in your code. https://www.sailogy.com/en/search/?search_where=ibiza&trip_date=2020-06-06&weeks_count=1&skipper=False&search_src=home only displays a single boat hence you're code is only returning one row.

Use https://www.sailogy.com/en/search/?search_where=ibiza&trip_date=2020-06-06&weeks_count=1&guests_count=&order_by=-rank&is_roundtrip=&coupon_code=&skipper=None instead in this case

You'll probally find use in adjusting the url parameters to filter boats at some point !

Samuel Middendorp
  • 652
  • 4
  • 14