What is the xpath for the Read more" link on this page to use with Selenium Python

Question

I am getting frustrated trying to find the xpath for the first "Read more" button in the review section of this website: https://www.tripadvisor.com/Hotel_Review-g60982-d209422-Reviews-Hilton_Waikiki_Beach-Honolulu_Oahu_Hawaii.html

I'm trying to scrape the reviews on my macbook, but keep getting an invalid syntax error. I was told that it's because the xpath isn't formatted correctly. All suggestions did not work.

I'm using selector gadget in Chrome, and this time I tried selecting an element with a div. This is what came out

//*[contains(concat( " ", @class, " " ), concat( " ", "eIVRK", " " ))]

Of course it did not work. Then I tried Chromes inspect element, and this is what it came up with

<div class="eIVRK" style="line-height: 20px;"><div class="dlJyA" data-test-target="expand-review"><span class="eljVo _S Z">Read more</span><span class="ui_icon caret-down zedZf"></span></div></div>

I have no clue what to do with this.

This also generated errors:

driver.find_elements(by=By.XPATH,'//*[@id="component_18"]/div/div[3]/div[3]/div[3]/div[3]/div[1]/div[2]/div').click()

File "/var/folders/6c/jpl964752rv_72zjclrp_8ym0000gn/T/ipykernel_24978/3907164188.py", line 8
    driver.find_elements(by=By.XPATH,'//*[@id="component_18"]/div/div[3]/div[3]/div[3]/div[3]/div[1]/div[2]/div').click()
                                                                                                                ^
SyntaxError: positional argument follows keyword argument

yang jiang · Answer 1 · 2022-03-14T01:25:21.667

0

I tested him and it worked

planA

driver.find_element_by_xpath('/html/body/div[2]/div[2]/div[2]/div[4]/div/div[1]/div[2]/div/div/div/div/div[2]/div[1]/div/div[1]/div[2]/div[2]/div/span').click()

planB

driver.find_element_by_xpath('//*[@id="COVID19"]/div/div[2]/div[1]/div/div[1]/div[2]/div[2]/div/span').click()

it is full code

from selenium import webdriver
from time import sleep


def test(key= 1):
    options = webdriver.ChromeOptions()
    options.binary_location = 'C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe'
    driver = webdriver.Chrome('D:\\python\\chromedriver.exe', options=options)

    driver.get('https://www.tripadvisor.com/Hotel_Review-g60982-d209422-Reviews-Hilton_Waikiki_Beach-Honolulu_Oahu_Hawaii.html')
    sleep(5)

    if key == 1:
        driver.find_element_by_xpath('/html/body/div[2]/div[2]/div[2]/div[4]/div/div[1]/div[2]/div/div/div/div/div[2]/div[1]/div/div[1]/div[2]/div[2]/div/span').click()
        sleep(10)
        driver.close()

    else:
        driver.find_element_by_xpath('//*[@id="COVID19"]/div/div[2]/div[1]/div/div[1]/div[2]/div[2]/div/span').click() 
        sleep(10)
        driver.close()
                                                
test(1)
test(2)

edited Mar 14 '22 at 01:25

answered Mar 14 '22 at 01:18

yang jiang

166
6

How did you come up with that path?I still got a bunch of errors.. I'll post the one I got in just a moment. – bandcar Mar 14 '22 at 01:30
You'd better set [chrome. Exe] and [chromedriver. Exe] as your own path before running my code – yang jiang Mar 14 '22 at 01:36
Thats the problem, I cannot find the correct xpath – bandcar Mar 14 '22 at 02:03
If you are new to XPath, you'd better learn it systematically: https://stackoverflow.com/questions/3030487/is-there-a-way-to-get-the-xpath-in-google-chrome – yang jiang Mar 14 '22 at 02:13
I found the xpath, but putting it into my code still gave an error. File "/var/folders/6c/jpl964752rv_72zjclrp_8ym0000gn/T/ipykernel_24978/3907164188.py", line 8 driver.find_elements(by=By.XPATH,'//*[@id="component_18"]/div/div[3]/div[3]/div[3]/div[3]/div[1]/div[2]/div').click() ^ SyntaxError: positional argument follows keyword argument – bandcar Mar 14 '22 at 02:37
I don't know your complete code. I only guarantee that I have solved the problem you raised. If you have a bigger problem, you should modify your problem description and attach your complete code – yang jiang Mar 14 '22 at 02:39
I got it sorted out by going back to safari as my webdriver instead of chrome. Thanks! @yangjiang – bandcar Mar 15 '22 at 18:37

score 0 · Answer 2 · answered Mar 14 '22 at 15:18

0

You can locate all the Read more link of the reviews within tripadvisor website using the following locator strategy:

Using XPATH:

//input[@placeholder='Search reviews']//following::div[@data-test-target='HR_CC_CARD']//div[@data-test-target='expand-review']/span

Snapshot:

What is the xpath for the Read more" link on this page to use with Selenium Python

2 Answers2