Timeout exception when using dev tools with selenium-java-4.0.0 and chromedriver v85

Question

I'm trying to use selenium dev tools java API, and for multiple API methods I'm getting java.util.concurrent.TimeoutException.

For example I'm trying to use Network.clearBrowserCache, which should work accroding to chromedriver docs: https://chromedevtools.github.io/devtools-protocol/tot/Network/

I'm calling clearBrowserCache using following code: chromeDriver.getDevTools().send(Network.clearBrowserCache())

It fails, but at the same time if I use other devTools commands like this: chromeDriver.getDevTools().send(Browser.getVersion())

It returns data properly.

Chrome version is: 85.0.4183.39
Chromedriver version is: 85.0.4183.87
Selenium-java version is: 4.0.0-alpha-6

mcernak · Answer 1 · 2020-09-14T17:56:00.660

Try calling createSession before calling clearBrowserCache.

Using your setup, this works:

chromeDriver.getDevTools().createSession();
chromeDriver.getDevTools().send(Network.clearBrowserCache())

and this produces java.util.concurrent.TimeoutException:

chromeDriver.getDevTools().send(Network.clearBrowserCache())

You can verify that the browser cache has been cleared with this snippet:

    ChromeDriver driver = new ChromeDriver();
    driver.get("https://refreshyourcache.com/en/cache-test/");
    Thread.sleep(2000);
    driver.getDevTools().createSession();
    driver.getDevTools().send(Network.clearBrowserCache());
    driver.get("https://refreshyourcache.com/en/cache-test/");
    Thread.sleep(5000);

If you run the code, the pages displayed in the test browser will show these images:

If you commment out the line driver.getDevTools().send(Network.clearBrowserCache()); then you get a different result:

Thank you, it fixes the original issue - 'clearBrowserCache' returns without error, but it doesn't do the job - I see that cache hasn't been cleared, maybe it's another unrelated issue, anyway maybe you have ideas how to fix that? — XZen, Sep 14 '20 at 08:12
I've added a code example for verifying the result of calling `clearBrowserCache()`. — mcernak, Sep 14 '20 at 17:57

undetected Selenium · Answer 2 · 2020-09-17T20:37:23.633

Using Selenium 4.0.0-alpha-6, Chrome v85 and ChromeDriver v85.0 through google-chrome-devtools you must be able to use getVersion() method as follows:

import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.browser.Browser;

public class BrowserGetVersion {
    public static void main(String[] args) {
        System.setProperty("webdriver.chrome.driver","C:\\WebDrivers\\chromedriver.exe");
        ChromeOptions options = new ChromeOptions();
        options.addArguments("--start-maximized");
        ChromeDriver driver = new ChromeDriver(options);
        DevTools devTools = driver.getDevTools();
        devTools.createSession();
        devTools.send(Browser.getVersion());
    }
}

Similarly, using the clearBrowserCache() method you should be able to clear the browser cache using the following code block:

import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.network.Network;

public class ClearChromeCache {
    public static void main(String[] args) {
        System.setProperty("webdriver.chrome.driver","C:\\WebDrivers\\chromedriver.exe");
        ChromeOptions options = new ChromeOptions();
        options.addArguments("--start-maximized");
        ChromeDriver driver = new ChromeDriver(options);
        DevTools devTools = driver.getDevTools();
        devTools.createSession();
        devTools.send(Network.clearBrowserCache());
        driver.get("https://www.google.com/");
    }
}

Additional Consideration

Additionally, you can also use setCacheDisabled(true) to completely disable the cache as follows:

Code Block:

import java.util.Collections;
import java.util.Optional;

import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.devtools.DevTools;
import org.openqa.selenium.devtools.network.Network;
import org.testng.Assert;
import org.testng.annotations.Test;

public class testngBasic {

    @Test
    public void foo() {
        System.setProperty("webdriver.chrome.driver","C:\\WebDrivers\\chromedriver.exe");
        ChromeOptions options = new ChromeOptions();
        options.addArguments("--start-maximized");
        options.setExperimentalOption("excludeSwitches", Collections.singletonList("enable-automation"));
        options.setExperimentalOption("useAutomationExtension", false);
        ChromeDriver driver = new ChromeDriver(options);
        DevTools devTools = driver.getDevTools();
        devTools.createSession();
        devTools.send(Network.clearBrowserCache());
        devTools.send(Network.enable(Optional.empty(), Optional.empty(), Optional.of(100000000)));
        devTools.send(Network.setCacheDisabled(true));
        devTools.addListener(Network.responseReceived(), responseReceived -> Assert.assertEquals(false, responseReceived.getResponse().getFromDiskCache()));
        driver.get("https://www.google.com/");

  }
}

This usecase

Possibly your code have nothing to do with java.util.concurrent.TimeoutException error and the real issue is either with the:

jdk version
guava version

Solution

Ensure that:

JDK is upgraded to current levels JDK 8u252.
guava is upgraded to guava-29.0-jre.

Outro

Disable cache in Selenium Chrome Driver

Thank you, creating session before using commands is the point I was missing, but unfortunately when I verify if cache is really cleared (by comparing number of requests being actually sent by browser to my server) I see that behavior is the same as if I didn't call 'clear browser cache' at all - requests seem to be cached. At the same time if I restart driver - I'm getting clear session & cache and my tests pass, so they are correct — XZen, Sep 14 '20 at 08:25
@XZen Checkout the answer update and let me know the status. — undetected Selenium, Sep 17 '20 at 20:38
Thank you, setCacheDisabled works as expected, but in my case I need to clear cache at some moment, not to completely disable it — XZen, Sep 18 '20 at 14:11
@XZen Great News !!! Glad to help you out !!! `setCacheDisabled` works per request basis. In the next line of code or operation it would be set as `disabled` again. Let me know the results of your test. By the mean time I will try to pull out something more catering to your usecase. — undetected Selenium, Sep 18 '20 at 14:19

score 2 · Answer 3 · answered Sep 12 '20 at 21:59

It worked fine

public void testCdt {
    final ChromeLauncher launcher = new ChromeLauncher();
    final ChromeService chromeService = launcher.launch(false);
    final ChromeTab tab = chromeService.createTab();
    final ChromeDevToolsService devToolsService = chromeService.createDevToolsService(tab);
    final Page page = devToolsService.getPage();
    Network network = devToolsService.getNetwork();
    // Clear browser cached
    network.clearBrowserCache();
    // Log requests with onRequestWillBeSent event handler.
    network.onRequestWillBeSent(
            event ->
                    System.out.printf(
                            "request: %s %s%s",
                            event.getRequest().getMethod(),
                            event.getRequest().getUrl(),
                            System.lineSeparator()));

    network.onLoadingFinished(
            event -> {
              chromeService.closeTab(tab);
              launcher.close();
            });
    network.enable();
    page.navigate("http://github.com");
    devToolsService.waitUntilClosed();
  }

Timeout exception when using dev tools with selenium-java-4.0.0 and chromedriver v85

3 Answers3

Additional Consideration

This usecase

Solution

Outro