9

Could anyone suggest headless browser for .NET that supports cookies and authomatically javascript execution?

Bogdan Dudnik
  • 413
  • 2
  • 7
  • 19
  • You mean the [WebBrowser control](http://msdn.microsoft.com/en-us/library/2te2y1x6.aspx)? – John Saunders Mar 06 '13 at 18:14
  • possible duplicate of [WebClient runs javascript](http://stackoverflow.com/questions/5372277/webclient-runs-javascript) – Chris Sinclair Mar 06 '13 at 18:15
  • How do you plan on using it? – jrummell Mar 06 '13 at 18:21
  • Sorry for my question is such poor. It is console crawler application for one web site. The problem is that the target web site use a lot of javascript generated cookies and big amount of javascript "on document ready redirections" so it is almost impossible to recreate session with cookies automatically (like getting cookies from specified url and go on) at next time. So i want to leave these "cookie generating/saving" and sequence of redirects for some headless browser (i don't need to render the html content, just work with it using htmlagilitypack). – Bogdan Dudnik Mar 06 '13 at 19:05

2 Answers2

9

Selenium+HtmlUnitDriver/GhostDriver is exactly what you are looking for. Oversimplified, Selenium is library for using variety of browsers for automation purposes - testing, scraping, task automation.

There are different WebDriver classes with which you can operate an actual browser. HtmlUnitDriver is a headless one. GhostDriver is a WebDriver for PhantomJS, so you can write C# while actually PhantomJS will do the heavy lifting.

Code snippet from Selenium docs for Firefox, but code with GhostDriver (PhantomJS) or HtmlUnitDriver is almost identical.

using OpenQA.Selenium;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium.Support.UI;

class GoogleSuggest
{
    static void Main(string[] args)
    {
        // driver initialization varies across different drivers
        // but they all support parameter-less constructors
        IWebDriver driver = new FirefoxDriver();
        driver.Navigate().GoToUrl("http://www.google.com/");


        IWebElement query = driver.FindElement(By.Name("q"));
        query.SendKeys("Cheese");
        query.Submit();

        WebDriverWait wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
        wait.Until((d) => { return d.Title.ToLower().StartsWith("cheese"); });

        System.Console.WriteLine("Page title is: " + driver.Title);

        driver.Quit();
    }
}

If you run this on Windows machine you can use actual Firefox/Chrome driver because it will open an actual browser window which will operate as programmed in your C#. HtmlUnitDriver is the most lightweight and fast.

I have successfully ran Selenium for C# (FirefoxDriver) on Linux using Mono. I suppose HtmlUnitDriver will also work as fine as the others, so if you require speed - I suggest you go for Mono (you can develop, test and compile with Visual Studio on Windows, no problem) + Selenium HtmlUnitDriver running on Linux host without desktop.

shturm
  • 857
  • 7
  • 18
4

I am not aware of a .NET based headless browser but there is always PhantomJS which is C/C++ and it works fairly well for assisting in unit testing of JS with QUnit.

There is also another relevant question here which might help you - Headless browser for C# (.NET)?

Community
  • 1
  • 1
Andrew Polaskey
  • 51
  • 3