0

I am trying to scrape this page with bs4 and I was wondering how can I scrape the EUR/USD, price change, and price % ?

I am pretty new to this, so this is all I have so far:

import requests
from bs4 import BeautifulSoup

url = 'http://www.cnbc.com/pre-markets/'
source_code = requests.get(url).text
soup = BeautifulSoup(source_code, 'lxml')

for r in soup.find_all('td', {'class': 'first text'}):
    print(r)
Martin Evans
  • 45,791
  • 17
  • 81
  • 97
qorka
  • 114
  • 1
  • 2
  • 12
  • What's the question? Is something not working as expected so that you can't continue, or are you asking us to write your actual code for you? – Mad Physicist Mar 16 '17 at 18:25
  • I think you need use requests.get(url, stream=True) and analyse code after. Look at [Bodycontent workflow](http://docs.python-requests.org/en/master/user/advanced/#body-content-workflow) – GeoStoneMarten Mar 16 '17 at 18:33
  • I guess my question is - is it possible to do? Looking thru the source code, it seems that the price, price change and % change are images? I could be (and probably am) completely wrong – qorka Mar 16 '17 at 19:24
  • @MadPhysicist after pulling the entire text from cnbc.com/pre-markets I noticed that the information (EUR/USD) does not show up - don't think it's possible to scrape.. – qorka Mar 17 '17 at 14:53

1 Answers1

2

The data you're looking for are probably loaded with javaScript and therefore you can't see them with bs4. But you can do it using an headless browser like PhantomJS, Selenium or Splash. See also this response: scraping dynamic updates of temperature sensor data from a website

Community
  • 1
  • 1
Tomark
  • 369
  • 2
  • 7