0

I want to extract some values from a website and the specific elements are

<div class="float_l dcMsg">
    <div class="float_l" style="margin-right: 5px; min-width: 105px;">Slow Stochastic(20,5)</div>
    <div class="float_l ind-color-box" style="margin-right: 5px; background: rgb(242, 38, 31);"></div>
    <div class="float_l" style="margin-right: 5px; min-width: 105px;">%K: 33.996</div>
    <div class="float_l ind-color-box" style="margin-right: 5px; background: rgb(0, 255, 0);"></div>
    <div class="float_l" style="margin-right: 5px; min-width: 105px;">%D: 18.393</div>
</div>

The values I want are on the 4th line(i.e. 33.996) and 6th line(i.e. 18.393).

These numbers I want are actually from a dynamic chart but I don't know whether it is from javascript. The numbers will update to the latest value after I pressed a certain button on the web page and the value of the numbers from the element will change as well accordingly. Besides, when I just hover the cursor over the chart, the numbers will change.

However the web page will not be reloaded but only the part of the numbers of the page element will be changed after I press the button.

I have tried this code but it returns [ ].

import urllib
import re

htmltext = urllib.urlopen("http://www.example.com").read()

regex = '<div class="float_l" style="margin-right: 5px; min-width: 105px;">(.+?)</div>'

pattern = re.compile(regex)

results = re.findall(pattern,htmltext)

print results

I have also tried using BeautifulSoup but it also returns [ ].

import bs4 as bs
import urllib

sauce = urllib.urlopen('http://www.example.com').read()

soup = bs.BeautifulSoup(sauce,'html.parser')

results = soup.findAll('div',style='margin-right: 5px; min-width: 105px;')

print results
kelvin
  • 43
  • 1
  • 1
  • 5
  • 2
    Based in the information you gave, you *might* want to use [selenium](http://stackoverflow.com/questions/17540971/how-to-use-selenium-with-python). BeautifulSoup does not handle dynamically loaded or changed values – Wondercricket Feb 10 '17 at 19:06
  • Ha! .... I don't have anything useful to contribute, but I need to say hi – Kelvin Feb 10 '17 at 21:16

1 Answers1

0

Selenium might be a good combination with this but its doable.

Maybe something like this: 

In [30]: for el in soup.findAll('div'):
    ...:     if el.has_attr('style') and 'margin-right: 5px' in el.attrs['style'] and el.attrs['class'] == ['float_l']:
    ...:         print el
    ...:
    ...:
<div class="float_l" style="margin-right: 5px; min-width: 105px;">Slow Stochastic(20,5)</div>
<div class="float_l" style="margin-right: 5px; min-width: 105px;">%K: 33.996</div>
<div class="float_l" style="margin-right: 5px; min-width: 105px;">%D: 18.393</div>
Kelvin
  • 1,357
  • 2
  • 11
  • 22
  • This doesn't return what I want. I think I would try to use selenium and thank you for your help anyway. – kelvin Feb 12 '17 at 15:52