1

I am trying to create a simple program to login to a webpage using my credentials and grab the total amount left of flex dollars I have in my account for college. Starting at the log in page, I log in, and am redirected to the page of interest, and I simply want to grab that dollar amount and perform some manipulation on it.

I am currently using webbot for the login portion of this, which works, I have just redacted the credentials:

from webbot import Browser

web = Browser()
web.go_to('insert my url here')
#enter your username and password in the into fields below
web.type('insert email here', into='username')
web.type('insert password here', into='password')
web.click('Login', tag='span')

This works perfectly so far, creating an instance of Chrome and logging into the page I want to grab the dollar amount from. I imagine I might want to proceed using urllib, however, I don't think urllib benefits from my current logged in instance of Chrome. How can I work around this and grab a simple html element from the page?

Kirency
  • 41
  • 6

1 Answers1

2

You first need to get the html source code for the current webpage. You can do that using get_page_source(). You then need to pass the html source code to beautifulsoup

from webbot import Browser
from bs4 import BeautifulSoup
import time

web = Browser()
web.go_to('insert my url here')
#enter your username and password in the into fields below
web.type('insert email here', into='username')
web.type('insert password here', into='password')
web.click('Login', tag='span')
time.sleep(5)

content = web.get_page_source()
soup = BeautifulSoup(content)

#You can now find the element you want
samples = soup.find_all("a", "item-title")
Caleb Njiiri
  • 1,563
  • 1
  • 11
  • 20
  • This needs to have a sleep function inserted in between the final web.click() and retrieving the html, otherwise it will retrieve the html from the login page due to the new page not loading yet – Kirency Dec 29 '19 at 02:42