Selenium - Wait for the webpage to request external URL

Question

I am basically trying to get a link to a m3u8 file. When the page first loads it requests the m3u8 file with a secret token that is changed everyday. The file is downloaded when the page if fully loaded. I can see it using chrome developer tools at network tab. Since this token is generated dynamically I need to let the website request this file first, then give me the URL to the file including the token (something like http://secret-website.com/file?token=342782g1bud1).

I have never used selenium, so I am wondering how I can do this if possible. I need to do it programmatically using python or c#.

http://stackoverflow.com/questions/15122864/selenium-wait-until-document-is-ready — Muckeypuck, Jan 10 '17 at 16:32
About "wait until page loaded". There are 2 different types in this. Let's say you're using Chrome, is the page still 'loading' if the circle in the tab has stopped spinning? Or do you mean that it's then ready? — Happy Bird, Jan 11 '17 at 15:36

score 1 · Accepted Answer · answered Jan 11 '17 at 15:47

I've found a solution which will wait untill it finds the .m3u8 file. I am using fiddlecore as proxy so I can see what is being sent by the browser. This will let me capture any request, and if the request contains .m3u8 it will simply print out the URL (which I need). Note that this won't work with https as fiddlecore needs certifications for that (I guess it is easy to fix). This is a simple working code.

   bool close = false;

    // - Initialize fiddler for http proxy

    Fiddler.FiddlerApplication.Startup(5000, true, false);
    Fiddler.FiddlerApplication.BeforeResponse += delegate(Session s)
    {
        string path = s.fullUrl;
        if (path.Contains("720p.m3u8"))
        {
            Console.WriteLine(path);
            close = true;
        }
    };

    // - Create a selenium proxy
    var seleniumProxy = new OpenQA.Selenium.Proxy { HttpProxy = "localhost:5000"};
    var profile = new FirefoxProfile();
    profile.SetProxyPreferences(seleniumProxy);

    // - Initialize selenium for browser automation
    IWebDriver driver = new FirefoxDriver(profile);
    driver.Navigate().GoToUrl("http://www.asite");
    while (!close) { }

    driver.Quit();
    Fiddler.FiddlerApplication.Shutdown();

Selenium - Wait for the webpage to request external URL

1 Answers1