1

I am trying to get statistics of a website. The 15 most recent drops. I want to be able to count the amount of drops that are of the milspec quality. I currently have code that counts how many of the 15 are blue but I want that number to keep going up everytime a new div that has the class of "item milspec" is added. Is there anyway to do this?

Here is my current code:

from time import sleep

from bs4 import BeautifulSoup
from selenium import webdriver

url = 'http://g2case.com/en'
browser = webdriver.Chrome()
browser.get(url)


def check_for_div_class_count(html, class_name):
    soup = BeautifulSoup(html, 'html.parser')

    milspecs = soup.findAll('div', {'class': class_name})
    return len(milspecs)

for i in range(10):
    print check_for_div_class_count(browser.page_source, 'item milspec')
    sleep(2)

browser.close()
L4undry
  • 281
  • 1
  • 4
  • 16
  • What do you mean by *keep going up everytime a new div that has the class of "item milspec" is added*? – Padraic Cunningham Apr 16 '16 at 21:27
  • @PadraicCunningham: I just checked the website. It refreshes the contents in page every few seconds. So I believe he wants to check the webpage every 2 seconds and count the new items. – James Apr 16 '16 at 21:29
  • @Legodog5678: If you want a running tally of how many items are 'item milspec', then you will need to modify your code. It currently counts the items on the page have 'item milspec', then you refresh the page every 2 seconds and count again. The problem is, depending on how quickly the webpage is pushing content, you might be either missing items that go by too quickly or double counting items that haven't been cleared from the page 2 seconds earlier. – James Apr 16 '16 at 21:38
  • Look into using explicit waits to wait for specific element count, example: http://stackoverflow.com/a/29941201/771848. – alecxe Apr 16 '16 at 22:00

0 Answers0