Trying to get song list on website doesn't work

Question

I have tried to use phantomjs , cheerio in node and webBrowser control in C# to get my song list , I can get the html successfully but without song list, I can't figure out why I can't get it...

The only way I can do is copy the html by dev tool and analyze it by Jquery.

Here is my code in WinForm :

  private void Form1_Load(object sender, EventArgs e)
    {
        webBrowser1.Navigate("http://grooveshark.com/#!/shinningstar1001/collection");
        webBrowser1.DocumentCompleted += webBrowser1_DocumentCompleted;
    }

    void webBrowser1_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
    {
        File.WriteAllText("D://test.txt", webBrowser1.DocumentText);
    }

In Cheerio :

var cheerio = require('cheerio');
var request = require('request');

var url = 'http://grooveshark.com/#!/shinningstar1001/collection';

request({
    url: url,
    headers: {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}
}, function (err, resp, body) {
    $ = cheerio.load(body);
    console.log(body);        
})

I guess it is because I can't get the full document after ajax load?

But why webBrowser Control can't work too? I can see full content is loaded in the control. Any advice will really appreciate.

I've tried @Murray Foxcroft solution still can't get the exact html which I want:

Additional question

By @Murray Foxcroft solution, I can get 8% of the list content, but why can't I get the full song list that pipe into the page? For example, I can get the song "Set me free" which is around 40th in list but can't get "This Love" which is around 70th in the song list. (Two song is on the site for sure)

        if (webBrowser1.ReadyState != WebBrowserReadyState.Complete)
            return;
        if (richTextBox1.Text.Length > 0) return;
        var songList = webBrowser1.Document.GetElementById("profile-grid");

        //try to get "This Love" that never step into the code:
        if (songList != null && songList.InnerHtml.Contains("This Love")){...}

        //"Set Me Free" is OK:
        if (songList != null && songList.InnerHtml.Contains("Set Me Free"))
        {
            richTextBox1.Text = songList.OuterHtml;                
        }        

Answer 1

For the WebBrowser sample, does the event actually fire?

Try associating the event before the navigate:

i.e. swap the lines to the following:

webBrowser1.DocumentCompleted += webBrowser1_DocumentCompleted;

webBrowser1.Navigate("http://grooveshark.com/#!/shinningstar1001/collection");

Also, DocumentCompleted may fire for every child document (like a CSS Style sheet), so make sure you are catching the event for the URL you are after.

void BrowserDocumentCompleted(object sender,
        WebBrowserDocumentCompletedEventArgs e)
{
  if (e.Url.AbsolutePath != (sender as WebBrowser).Url.AbsolutePath)
    return; 

  //The page is finished loading 
}

Further details here: Detect WebBrowser complete page loading

Final solution - the content is piped in to the main page from another source so looking for the target div is about the best solution:

 private void webBrowser1_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
        {
            // If the ReadyState is Complete then the page or an iFrame within have completed downloading.  
            if (webBrowser1.ReadyState != WebBrowserReadyState.Complete)
                return; 

            // Ensures only the first match of page-content is resturned to the RichTextBox.
            // If this does not contain what you are looking for then you may need to find an 
            // additional way to refine for the content you are after. 
            if (richTextBox1.Text.Length > 0) return;

            // Check to see if we have got the page-content div in our result source 
            // and set the richtextbox if we have it.
            var songList = webBrowser1.Document.GetElementById("page-content");
            if (songList != null)
            {
                richTextBox1.Text = songList.OuterHtml;
            }
        }