Good morning. I am developing a spider to review a few web pages. I can't do it without using Selenium. But the problem with Selenium is that it consumes a lot of resources and is slow. I am looking for the optimization way. From what I see the main problem is that Selenium loads the entire website, with all its resources. But I just need javascript and html to work for me. But I don't need images. Can I somehow prevent images from loading in the Selenium browser in C #?
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
using (IWebDriver driver = SeleniumUtility.GetChromeDriverHidden())
{
driver.Url = "https://stackoverflow.com/";
string html = driver.PageSource;
}
internal static ChromeDriver GetChromeDriverHidden(bool hidden = true)
{
ChromeDriverService service = ChromeDriverService.CreateDefaultService(".");
service.HideCommandPromptWindow = true; // Hide output commands in console
var options = new ChromeOptions()
{
AcceptInsecureCertificates = true // This lets the browser accept the insecure certificate. Set hidden = false
};
if (hidden)
{
options.AddArgument("headless"); // hide window if added to options
}
return new ChromeDriver(service, options);
}
I see one solution, but in C# I don't understand how to do it.