How to scrape Booking.com with Python and BeautifulSoup

Asked Sep 04 '19 at 15:51

Active Sep 04 '19 at 15:51

Viewed 1,071 times

I want to do a search on booking, and take the price of the first hotel, as this.

This is the part of the code that interests me

<div class="prco-ltr-right-align-helper">
<div class="bui-price-display__value prco-inline-block-maker-helper" aria-hidden="true">
€&nbsp;702
</div>
<span class="bui-u-sr-only">
Prezzo
€&nbsp;702
</span>
</div>

This is my simple code in python

import requests
main_url = the link above
req = requests.get(main_url)
text = req.text
print(text.find('bui-u-sr-only'))
print(text.find('bui-price-display__value prco-inline-block-maker-helper'))

The two strings you're looking for are not present, why?

asked Sep 04 '19 at 15:51

Onire77

That site appears to be loaded asynchronously, meaning that the full page doesn't load all at the same time. You may need browser automation like Selenium to access it. – G. Anderson Sep 04 '19 at 15:53
Try chrome headless option along with selenium. – Sureshmani Kalirajan Sep 04 '19 at 16:05
This looks like what you're trying to do. https://stackoverflow.com/questions/45448994/wait-page-to-load-before-getting-data-with-requests-get-in-python-3 – Cathal Cronin Sep 04 '19 at 16:05

How to scrape Booking.com with Python and BeautifulSoup

0 Answers0