Is it possible to grab source code from a web page and change it with selenium + python?

Question

I'm trying to click on a radio button in order to get the add new page section to show up. However the error code I keep getting is AttributeError: 'WebElement' object has no attribute 'find_element_by_class'

I was able to find the button by class = 'button' but when I use .click() I keep getting that error. I understand if I got a non-clickable error.

I did notice that the aria-checked element is set to = Flase when the radio button hasn't been clicked on, and it changes to = True when it has. So i'm guessing that's what I'm going to have to do to solve this.

So my question is using python + selenium, how can I get this to change to true so it shows the rest? I am still new to codeing so I hope I explained my situation as best as I could, I finally figured out about Shadow DOM.

This is my code

from selenium import webdriver

def expand_shadow_element(element):
    shadow_root = cdriver.execute_script('return arguments[0].shadowRoot', element)
    return shadow_root
#chrom driver
cdriver = webdriver.Chrome(executable_path='C:\\Users\\name\Downloads\\chromedriver.exe')
#open up page to chrome settings.
cdriver.get('chrome://settings/')

root1 = cdriver.find_element_by_tag_name('settings-ui')
shadow_root1 = expand_shadow_element(root1)

root2 = shadow_root1.find_element_by_id('main')
shadow_root2 = expand_shadow_element(root2)

root3 = shadow_root2.find_element_by_tag_name('settings-basic-page')
shadow_root3 = expand_shadow_element(root3)

root4 = shadow_root3.find_element_by_tag_name('settings-on-startup-page')
shadow_root4 = expand_shadow_element(root4)

root5 = shadow_root4.find_element_by_tag_name('controlled-radio-button')
shadow_root5 = expand_shadow_element(root5)

#error code herer code
root6 = shadow_root5.find_element_by_class('disc-wrapper').click()

Can you please provide the code you are currently using that gives you that error message. — NoodleBeard, Nov 15 '19 at 16:20
it has to be `find_element_by_class_name()` - with `_name` at the end. — furas, Nov 15 '19 at 16:43

score 0 · Answer 1 · edited Nov 15 '19 at 16:55

you can do the following in java, you cannot change the code, since, most of the websites are dynamically created and not statically rendered.

WebDriver driver = new ChromeDriver();
driver.get("https://www.googel.com/");
String str = driver.getPageSource();
System.out.println(str);

am sure the same can be done in python as well,

from selenium import webdriver

options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument("--test-type")
options.binary_location = "/usr/bin/chromium"
driver = webdriver.Chrome(chrome_options=options)
driver.get('https://python.org')
driver.getPageSource();

Is it possible to grab source code from a web page and change it with selenium + python?

1 Answers1