-2

I am trying to extract Quote data from the CNBC US Market Movers page.

Using BeautifulSoup4, I tried this simple look at the page.

from bs4 import BeautifulSoup
import requests

website = requests.get('https://www.cnbc.com/us-market-movers/')
soup = BeautifulSoup(website.text, 'lxml')

print(website.text)

The printed result doesn't contain any quotes or stocks or anything.

What I am looking for are these symbols.... enter image description here

Quentin
  • 914,110
  • 126
  • 1,211
  • 1,335
TheBigDuck
  • 9
  • 4
  • 1
    Does this answer your question? [How to call JavaScript function using BeautifulSoup and Python](https://stackoverflow.com/questions/48603339/how-to-call-javascript-function-using-beautifulsoup-and-python) – Quentin May 18 '21 at 07:38

1 Answers1

0

You can find this data by using chrome in developer mode and go to network profile and refresh you website

then go to xhr tag find your particular data by different links and as from image you can see i have find that json data now you can copy link address and work with json data

import requests
res=requests.get("https://gdsapi.cnbc.com/market-mover/groupMover/SP500/CHANGE_PCT/BOTH/12.json?source=SAVED&delayed=false&partnerId=2")
data=res.json()
for d in data['rankedSymbolList']:
    for main_data in d['rankedSymbols']:
        print(main_data['symbolDesc'])

Output:

Seagate Technology PLC
Western Digital Corp
Occidental Petroleum Corp
Baker Hughes Co
Newmont Corporation
...

Image: enter image description here

Bhavya Parikh
  • 3,304
  • 2
  • 9
  • 19
  • Clearly, BeautifulSoup is powerful, but it can't do everything. Using the XHR tags to find the data stream, using Preview, and seeing the various values has been an eye opener. Thank you for your answer. – TheBigDuck May 18 '21 at 12:16
  • Yes @TheBigDuck i also learn this from stackoverflow and its easy to use! – Bhavya Parikh May 18 '21 at 12:20