Accepting cookies with Selenium in Java

Question

I am trying to scrape a news webside, but it is not possible to accept the "accept cookies" popup by using the click() method. I can see the button in my HTML code in my browser, but when I use getPageSource() method the code for the button is not included.

Here my code block

public class Webscraping {
  public static void main(String[] args) throws Exception{
    System.setProperty("webdriver.chrome.driver","C:\\Users\\Marvin\\Desktop\\Webscraping\\chromedriver.exe");
    
    //Pop Up blocken
    //ChromeOptions options = new ChromeOptions();
    //options.addArguments("disable-popup-blocking");
    
    WebDriver driver = new ChromeDriver();
    driver.manage().timeouts().implicitlyWait(5, TimeUnit.SECONDS);
    String url = "https://www.focus.de/";
    driver.get(url);
    Thread.sleep(5000);
    //HTML Code print
    System.out.println(driver.getPageSource());
  }
}

score 0 · Answer 1 · answered Dec 12 '21 at 17:10

0

To click() on the element Akzeptieren within the url https://www.focus.de/ as the the desired element is within a <iframe> so you have to:

Induce WebDriverWait for the desired frameToBeAvailableAndSwitchToIt.
Induce WebDriverWait for the desired elementToBeClickable.

You can use either of the following Locator Strategies:

Using cssSelector:

driver.get("https://www.focus.de/");
new WebDriverWait(driver, 10).until(ExpectedConditions.frameToBeAvailableAndSwitchToIt(By.cssSelector("iframe[title='Iframe title']")));
new WebDriverWait(driver, 10).until(ExpectedConditions.elementToBeClickable(By.cssSelector("button[title='Akzeptieren']"))).click();

Using xpath:

driver.get("https://www.focus.de/");
new WebDriverWait(driver, 10).until(ExpectedConditions.frameToBeAvailableAndSwitchToIt(By.xpath("//iframe[@title='Iframe title']")));
new WebDriverWait(driver, 10).until(ExpectedConditions.elementToBeClickable(By.xpath("//button[@title='Akzeptieren']"))).click();

Reference

You can find a couple of relevant discussions in:

answered Dec 12 '21 at 17:10

undetected Selenium

183,867
41
278
352

Creating multiple instances of `WebDriverWait` is not a good practice for the same purpose. for this small use case particularly not. Create instance once and then assignee it's a reference to a variable and uses a reference variable instead. – cruisepandey Dec 12 '21 at 18:27
@cruisepandey _Creating multiple instances of WebDriverWait is not a good practice_: where did you come across all these? What's wrong if at all putting the _GC_ at work? – undetected Selenium Dec 13 '21 at 08:57
When you write `new` in JAVA, In your system RAM there is a specific memory allocation named heap. The moment you write `new` an object will be created. To make heap memory optimization one should not create multiple objects for the same _purpose_. It is called _space complexity_. You can read more about it in the computer design & Algos section. – cruisepandey Dec 13 '21 at 09:26
@cruisepandey You haven't answered my question, why not to put _GC_ at work? What else do you expect _GC_ to do? I would have loved to explain to you more about memory management and how _GC_ works. Perhaps that would be out of scope for this question. – undetected Selenium Dec 13 '21 at 09:30
Why would you want to put things to GC, when you can handle it through your program? It makes Java programs efficient memory-wise, but there is no need to, and by simply following best practices. The code that you have given is not efficient at all, how do you make that efficient? You are creating two objects unnecessary for a single purpose. I don't understand that. – cruisepandey Dec 13 '21 at 09:37
you got it all wrong. I am not abusing your answer. The intention was to understand why you did that? Unfortunately, you've mentioned a lot of things but have not answered my question. Anyway, peace out! – cruisepandey Dec 13 '21 at 13:38
1

@cruisepandey Gimme some time I'll explain. – undetected Selenium Dec 13 '21 at 15:26

Accepting cookies with Selenium in Java

1 Answers1

Reference