1

The web application we use for tracking field logs is B2W. The field logs are listed in a table wrapped in an auto-scrolling div. I am using vba to get the field log info from that table to store in Access. But it will only retrieve the first 25 rows. I've tried using

driver.ExecuteScript(window.ScrollTo())

but it doesn't work for me. If I scroll, it tells me the elements can no longer be found. If I use FireBug, I can see that the data is there in between the

<span>This would be a field log id number</span>

I am assuming it's not there until I scroll to it. I've been googling but nothing I've tried has worked. My current code is:

Private selDriver as New Selenium.ChromeDriver

'I have also tried using FirefoxDriver (I am using Firefox 46.0.1)
'Same results

Public Sub AppendFieldLogs()
Dim db As DAO.Database
Dim rs As DAO.Recordset
Set db = CurrentDb
Set rs = db.OpenRecordset("tblFieldLogs")
Dim i As Integer
With rs
    Dim FLogs As Object, FLog As Object, FLogData As Object
    With selDriver
        .Get "http://tls-okc-b2w/B2W/FieldLogListing.aspx"
        .Wait 600
    End With
    selDriver.ExecuteScript ("window.scrollTo(0, document.body.scrollHeight);")
    For Each FLogs In selDriver.FindElementsByXPath("/html/body/div[1]/div/form/div[2]/div[3]/div/div/div/div/div[2]/div[2]/div[3]/div[1]/table/tbody")
        For i = 1 To FLogs.FindElementsByXPath(".//tr").Count
            If FLogs.FindElementByXPath(".//tr[" & i & "]").IsPresent Then
                Set FLog = FLogs.FindElementByXPath(".//tr[" & i & "]")
                If FLog.FindElementByXPath(".//td[2]").Text = "" Then
                    Exit For
                End If
                .AddNew
                !FieldLogID = FLog.FindElementByXPath(".//td[2]").Text
                !FieldLogDate = FLog.FindElementByXPath(".//td[3]").Text
                !FieldLogSupervisor = FLog.FindElementByXPath(".//td[4]").Text
                !FieldLogStatus = FLog.FindElementByXPath(".//td[6]").Text
                !FieldLogJob = FLog.FindElementByXPath(".//td[7]").Text
                .Update
                .Bookmark = .LastModified
            Else
                Exit For
            End If
        Next i
    Next FLogs
End With
Set rs = Nothing
Set db = Nothing
selDriver.Quit
DoCmd.OpenTable "tblFieldLogs"
End Sub

This works for the first 25 rows. After that, it quits and doesn't give any errors. It just doesn't fetch the data.

EDIT: Here is a link to my onedrive where I saved some of pages HTML and ASPX code. OneDrive

EDIT: ALMOST solved this myself: I found a way to scroll the additional rows into view with the .scrollIntoView method. I am using an if statement to check if the value of the column = "", if it does, then use

driver.findelementmethodofyourchoice.scrollIntoView

and it works for the most part! It will only grab the first 200 rows, but I can see in the source code for the page that only the first 200 rows are loaded into the table, but as you scroll more, more will load. I just need to figure out how to get that 'more' to load.

AmandaKay10
  • 25
  • 6
  • Have you tried scrolling to an element at the bottom of the page? E.g. https://stackoverflow.com/a/49840292/6241235 Or find a counter than can be used to govern scrolling e.g. https://stackoverflow.com/a/50846524/6241235 – QHarr Aug 16 '18 at 21:22
  • Yes I have tried both those things, the table claims it only goes to row 200, but their are much more than that. But when I manually scroll down to the last row in the table, it says the tr[200]. No matter what I use as a loop counter it only grabs the first 25 rows. This is B2W, installed on a server out of our main office. We access it through a browser like we would any other webpage. But it's not accessible outside our network, so there's no way anyone but me can test any of this unless they use B2W also. – AmandaKay10 Aug 17 '18 at 21:46
  • Yeah I realised from looking at the script so didn't even bother asking for an URL. Are 25 rows the visible number in the initial scroll view? What is the significance of the 25 figure? And it sounds like you are saying you can automate scrolling down to the very bottom but if you try and retrieve rows during this process then the browser quits? – QHarr Aug 18 '18 at 06:13
  • No it will not automate scrolling, it tells me the method doesn't exist. Maybe I'm using the wrong syntax. But if I manually scroll in the screen while the script is running, it says the element no longer exists and quits. I have no idea why 25 is the number it retrieves. I can visibly see more than 25 rows on the screen without even needing to scroll. The page automatically expands and adds more rows AS you scroll down, infinitely, but the last row always appears as tr[200] even if their are 20000 rows. – AmandaKay10 Aug 20 '18 at 14:43
  • Is there js appending rows in blocks? – QHarr Aug 20 '18 at 14:48
  • I'm not 100% certain, but that is what I think is happening, or something similar. I edited my post with a link to my OneDrive with the FieldLogListing page's ASPX and also the TXT file of the HTML I copied and saved from FireBug. – AmandaKay10 Aug 20 '18 at 14:55
  • I also face a similar issue and now I'm trying to iterate through rows. But still have the same issue and now planning to scroll line by line to make it load and then copy the data. Will that work? – Shyam Sundar Shankar Dec 14 '21 at 11:15

0 Answers0