Can't execute my code without "on error resume next"

Question

Writing a crawler in VBA in combination with selenium to parse the price of different products from a webpage, I get an issue upon execution. It breaks when it finds certain prices are none. Using "on error resume next" I can get the full results. However, I wish to execute my code without using "on error resume next". If it were not for selenium, I could have used Length property to get rid of that. But, selenium doesn't support that. Hope I can have any workaround here.

Sub Redmart_scraping()
Dim driver As New ChromeDriver
Dim posts As Object, post As Object

With driver
    .get "https://redmart.com/bakery"
    Set posts = .FindElementsByCss("li.productPreview")
End With

On Error Resume Next

For Each post In posts
    i = i + 1
    Cells(i, 1) = post.FindElementByCss("span[class^=ProductPrice__price]").Text
Next post
End Sub

Answer 1

You can extract the prices as follows:

Sub Redmart_scraping()
Dim driver As New ChromeDriver
Dim posts As Object
Dim i As Long

With driver
    .get "https://redmart.com/bakery"
End With

Columns("A:A").NumberFormat = "[$$-409]#,##0.00"

For Each posts In driver.FindElementsByClass("productPreview")
    i = i + 1
    'Cells(i, 2) = posts.Text
    For Each Item In Split(posts.Text, vbLf)
        If InStr(1, Item, "$", vbTextCompare) > 0 Then
            If InStr(2, Item, "$", vbTextCompare) > 0 Then
                Cells(i, 1) = Mid(Item, 2, InStr(2, Item, "$", vbTextCompare) - 2)
            Else
                Cells(i, 1) = Right(Item, Len(Item) - 1)
            End If

       End If
    Next
Next
End Sub

Please note that posts.Text holds all the information you need for one item. So besides price, you can extract item name, discount, customer ratings, weight, price before discount, and guaranteed fresh dates.. Uncomment 'Cells(i, 2) = posts.Text and see for yourself.

I am leaving rest of the fun work to you. Good luck!

Answer 2

As it seems, from looking at accepted answer, your script fails on the promo prices use 2 attribute selectors combined with OR to set your original posts variable. Then you have no nulls to handle. (Belated answer - came across this whilst hunting for a resource!).

Option Explicit
Public Sub Redmart_scraping()
    Dim driver As New ChromeDriver
    Dim posts As Object, post As Object, i As Long

    With driver
        .get "https://redmart.com/bakery"
        Do
        Set posts = .FindElementsByCss("li.productPreview span[class^=ProductPrice__price],span[class^=ProductPrice__promo_price]")
        Loop While posts.Count = 0
    End With

    For Each post In posts
        i = i + 1
        Cells(i, 1) = post.Text
    Next post
End Sub

Answer 3

According to the documentation, this is how Florent B. suggested to avoid any index error and make the code run through the end.

Sub ScrapePrice()
    Dim driver As New ChromeDriver, post As Object

    With driver
        .get "https://redmart.com/bakery"
        For Each post In .FindElementsByCss("li.productPreview")
            If Not post.FindElementByCss("span[class^='ProductPrice__price']", Raise:=False, timeout:=0) Is Nothing Then
                R = R + 1: Cells(R, 1) = post.FindElementByCss("span[class^='ProductPrice__price']").Text
            End If
        Next post
    End With
End Sub

Answer 4

Sub Testing()
    Dim driver As New WebDriver
    Dim posts As Object, post As Object
    dim this as string, that as string

    Set driver = New WebDriver
    driver.Start "Phantomjs", "https://redmart.com"
    driver.get "/bakery"
    Set posts = driver.FindElementsByXPath("//div[@class='productDescriptionAndPrice']")
    On Error Resume Next
    For Each post In posts
        i = i + 1
        If Isnull(post.FindElementByXPath(".//h4/a").Text) = True Then
            this = ""
        Else
            this = post.FindElementByXPath(".//h4/a").Text
        End if
        Cells(i, 1) = this

        if IsNull(post.FindElementByXPath(".//span[@class='ProductPrice__price___3BmxE']").Text) = True then
            that = ""
        else
            that = post.FindElementByXPath(".//span[@class='ProductPrice__price___3BmxE']").Text
        end if
        Cells(i, 2) = that
    Next post
End Sub

The age old "null pointer" nuisance. Checking for nulls is important. There's prolly a better way to do this but you get the point (pun intended)

Answer 5

On Error Resume Next just skips any runtime errors - you are right in not wanting to apply this in general.

If you cannot prevent these errors from occurring in the first place (by testing for null, for example) you should activate On Error Resume Next only for that part of the code that actually can create these errors and deactivate it right after with On Error Goto 0.

Inside of these On Error blocks you can also explicitly test which error occurred (and hence will be ignored) by using the Err object. This way you could re-raise all errors that you didn't expect and only ignore those you expected. Beware: Raising an error must be done outside of an On Error block, but leaving the block will reset the Err object. Therefore you would have to save all necessary error information in another variable (or variables) to access it after you leave the On Error block.

I'm afraid with the general messed up design of error handling provided by VBA, there is no better way.