VB 2010: How can I get the content of a specific span in a specific class of a webpage?

Question

Let me explain: I want to make a currency converter form that uses online rates so that it's up to date. I plan to use Google, which has a currency converting function built in - type in:

[Any currency symbol][Any numeric amount] + in + [Any currency symbol]

---

I know how to format the URLs I will be searching for, and which class the span (whose text) I want is in, I simply just don't know how to programmatically take the result out and use it in my form.

Here's the applicable HTML code from a "£1 in $" conversion:

<div class="vk_ans vk_bk curtgt" style="padding-bottom: 4px">
<span style="word-break:break-all">1.68</span>
<span>US Dollar</span></div>

The class is called:

vk_ans vk_bk curtgt

The span text is the first one in the class (the one that contains "1.68")

BTW, I completely comprehend that there are easier-to-use API websites for this purpose, but I want to use Google because:

1. It will always be up
2. It's a good chance for me to learn how to grab a specific part a webpage.

Answer 1

I personnaly use http://www.nuget.org/packages/ScrapySharp/2.2.63 for html-scraping, it allows use to use CSS3 selectors to nicely grab what you need.

In your case it would be as simple as this :

Dim doc as HtmlAgilityPack.HtmlDocument = new HtmlAgilityPack.HtmlDocument()
doc.LoadHtml(GetRawHtml(theUrl))
Dim body as HtmlNode= doc.DocumentNode.SelectSingleNode("//body")
Dim yourSpan as HtmlNode = body.CssSelect(".vk_ans.vk_bk.curtgt").First()
dim yourValue as Double = Double.Parse(yourSpan.InnerText)


 Function GetRawHtml(url As String) As String
    Dim html As String = String.Empty
    Dim request As WebRequest = WebRequest.Create(url)

    Try
        Using response As WebResponse = request.GetResponse()
            Using data As Stream = response.GetResponseStream()
                Using sr As New StreamReader(data)
                    html = sr.ReadToEnd()
                End Using
            End Using
        End Using
    Catch e As Exception
        yourLogger.Error("WebRequest failed at url `{0}`. Error: {1}", url, e.ToString())
    End Try

    Return html
End Function

Also if the request is slow, you might want to change proxy attribute to nothing, see here : HttpWebRequest is extremely slow!