How to scrape the real time stock price quote using Selenium Python

Question

I need to write a small app to get the real-time quote on Money18.on.cc for my school project. I'm aiming get all the data shown . I have tried Selenium w/ a list of tickers ["700", "3690", "1"], but the outcome looks strange, e.g. sometimes all 3 results are the same or the order of the corresponding price is wrong. Here is my code:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

webdriverPath = getFileContentItem("all_file_path.txt", "chromeWebdriverDir")
s = Service(webdriverPath)
driver = webdriver.Chrome(service = s)
driver.implicitly_wait(10)
driver.set_page_load_timeout(20)
print("Successful open browser...")


tickerList = ["700", "3690", "1"]
money18Link = "https://money18.on.cc/eng/info/hk/liveinfo_quote_00700.html"
print(f"money18Link:\n{money18Link}")
driver.get(money18Link)
driver.find_element(By.XPATH, '//div[@class="industryClose"]/img').click() # remove the AD from the main window

for ticker in tickerList:
    driver.get(money18Link)
    inputElem = driver.find_element(By.CSS_SELECTOR, '[class="stockInput stock-number"]')
    inputElem = myElem = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.CSS_SELECTOR, '[class="stockInput stock-number"]')))
    inputElem.click()
    inputElem.clear()
    inputElem.send_keys(ticker)
    inputElem.send_keys(Keys.ENTER)

    print(f"ticker: {ticker} is running...")
    driver.switch_to.default_content()
    curPx = driver.find_element(By.XPATH, '//div[@class="section-body"]/div[4]/div[1]/div/div/div[1]/span[@class="value"]').text
    print(f"curPx: {curPx}")

A few result scenarios below but look strange, the stock price scrapped is not corresponding to its ticker:

Scen 1:
ticker: 700 is running...
curPx: 374.400
ticker: 3690 is running...
curPx: 374.400
ticker: 1 is running...
curPx: 374.400

Scen 2:
ticker: 700 is running...
curPx: 374.400
ticker: 3690 is running...
curPx: 374.400
ticker: 1 is running...
curPx: 155.200

score 0 · Accepted Answer · answered Apr 16 '22 at 12:17

You don't need to use selenium as there is a JSON API.

Take a look a this example:

import time

import requests

headers = {
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36 OPR/83.0.4254.62',
}

api_url = 'https://realtime-money18-cdn.on.cc/securityQuote/genStockDetailHKJSON.php'

stockcodes = ['700', '3690', '1']

while True:
    for code in stockcodes:
        params = {'stockcode': code}

        r = requests.get(api_url, headers=headers, params=params)
        data = r.json()
        price = data['real']['ltp']
        print(f'Price for code {code} = {price}')

    print()
    time.sleep(5)

Here is the output:

Price for code 700 = 374.400
Price for code 3690 = 155.200
Price for code 1 = 56.750

Thx Alex, it works perfectly. Could you give me more color as to how to determine a website which has a JSON API? and how to get the api url? Coz, I tried to find the api url, but no luck. Many thx — user18823104, Apr 18 '22 at 05:25
You could use Web Dev Tools. In Chromium-like browser its `Ctrl+Shift+C`. Find a `Network` tab there, usually, all APIs goes into `fetch/xhr` tab, so, just check all of them by viewing at their responses — Alex Kosh, Apr 18 '22 at 09:42

undetected Selenium · Answer 2 · 2022-04-16T15:43:25.463

To print the stock_number and the stock_value you can use the following Locator Strategies:

Using CSS_SELECTOR:

driver.get("https://money18.on.cc/eng/info/hk/liveinfo_quote_00700.html")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div.industryClose>img"))).click()
stock_number = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.CSS_SELECTOR, "input.stockInput.stock-number"))).get_attribute("value")
stock_value = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.CSS_SELECTOR, "div.text-right.stock-price span.value"))).text
print(f"{stock_number} is {stock_value}")

Using XPATH:

driver.get("https://money18.on.cc/eng/info/hk/liveinfo_quote_00700.html")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[@class='industryClose']/img"))).click()
stock_number = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, "//input[@class='stockInput stock-number']"))).get_attribute("value")
stock_value = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, "//div[@class='text-right stock-price']//span[@class='value']"))).text
print(f"{stock_number} is {stock_value}")

Console Output:
```
00700 is 374.400
```

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Thanks, quick one, have you run your code using a for loop/etc. with a ticker list ['700', '3690, '1']? My pb is if I run in a loop with a ticker list, the stock price of the 1st ticker will repeat showing in the 2nd and even the 3rd ticker as shown in the questions I posted above i.e. Scen 1 & Scen 2. I just put your code into a for loop w/ a ticker list, same strange outcomes. Could you give me more insight on this? Thx a lot — user18823104, Apr 18 '22 at 05:18
Cool, it seems you have got an accepted answer. All the best. — undetected Selenium, Apr 18 '22 at 06:04

How to scrape the real time stock price quote using Selenium Python

2 Answers2