How to interact with Developer tools in chrome using selenium or any other library

Question

I am trying to read or interact with network and console tabs of developer tools in chrome. Could you please guide me how to achieve this.

Thanks

Answer 1

The short answer is no. How to open Chrome Developer console in Selenium WebDriver using JAVA. As the provided link states you cannot directly access the chrome developer tools.

But if you are interested in access the contents of the browser console and network tab, selenium provides you a way.

System.setProperty("webdriver.chrome.driver", getChromeDriverLocation());

LoggingPreferences loggingprefs = new LoggingPreferences();
loggingprefs.enable(LogType.BROWSER, Level.WARNING);
loggingprefs.enable(LogType.PERFORMANCE, Level.WARNING);

DesiredCapabilities capabilities = DesiredCapabilities.chrome();
capabilities.setCapability(CapabilityType.LOGGING_PREFS, loggingprefs);

driver = new ChromeDriver(capabilities);

You can then print the logs as needed

LogEntries logEntries = SeleniumBaseTest.getWebDriver().manage().logs()
            .get(org.openqa.selenium.logging.LogType.BROWSER);
for (LogEntry entry : logEntries) {
    System.out.println((String.format("%s %s %s\n", new Date(entry.getTimestamp()), entry.getLevel(),
                entry.getMessage())));
}

LogType.BROWSER will give you the browser console. Logtype.PERFROMANCE will give you the network tab.

Other ways to access the network tab is to use a browser proxy to record the transactions. http://www.seleniumeasy.com/selenium-tutorials/browsermob-proxy-selenium-example

Answer 2

In Python, Pychrome is working fine as an interface to DevTools Protocol Viewer.

Below is an example I used with a mix of Selenium for the main request and Pychrome as I wanted to get Images without downloading them twice...

import base64
import pychrome
def save_image(file_content, file_name):
    try:
       file_content=base64.b64decode(file_content)
       with open("C:\\Crawler\\temp\\" + file_name,"wb") as f:
            f.write(file_content)
    except Exception as e:
       print(str(e))

def response_received(requestId, loaderId, timestamp, type, response, frameId):
    if type == 'Image':
        url = response.get('url')
        print(f"Image loaded: {url}")
        response_body = tab.Network.getResponseBody(requestId=requestId)
        file_name = url.split('/')[-1].split('?')[0]
        if file_name:
            save_image(response_body['body'], file_name)


tab.Network.responseReceived = response_received

# start the tab 
tab.start()

# call method
tab.Network.enable()

# get request to target the site selenium 
driver.get("https://www.realtor.com/ads/forsale/TMAI112283AAAA")

# wait for loading
tab.wait(50)