0

We have created a program that monitors domain/url info of the Firefox browser, so every few seconds this program will grab the browser address bar information using UI Automation (similar to here Get active Tab URL in FireFox with C++). When this happens, it causes a performance slowdown for the user when that person attempts interaction with Firefox, such as typing in characters in a control field, switching between tabs (if multiple tabs open), etc. Using Task Manager Resource Monitor, I can see sustained "plateau spikes" in CPU usage when the UI Automation "fires" (attempts to grab the URL info). So my question: Is this slow down in performance likely just caused by the UI Automation iterating thru the browser page, or is there some sort of conflict with web page interaction (by the user) and the UI Automation (running within the monitor program)?

Example UI Auto code:
    QString firefoxqstr = "Search with Google or enter address";

if(title.contains("mozilla firefox", Qt::CaseInsensitive))
    {
        addressBarIdentifier = (wchar_t*)firefoxqstr.utf16();
        //addressBarIdentifier = (LPWSTR)(L"Search with Google or enter address");
        getBrowserDomain(3,topWindow, addressBarIdentifier, title, domain, errqstr);
}



void getBrowserDomain(int xtrytype, HWND topWindow, LPWSTR addressBarIdentifier, QString title, QString& domain, QString& errqstr)
{
        errqstr = "blank";    
        // It's a Non IE browser.  Get the URL using addressBarIdentifier
    // main UIAutomation interface
    IUIAutomation *uiauto;

    // initial value assigned to UIAutomation
    uiauto = NULL;
    // Initialing COM functionality
    // My approach uses UI Automation
    CoInitialize(NULL);
    HRESULT hr =
            CoCreateInstance(__uuidof(CUIAutomation),
                             NULL,
                             CLSCTX_INPROC_SERVER,
                             __uuidof(IUIAutomation),
                             (void**)&uiauto);

    // Normally, this shouldn't happen but just in case
    if(FAILED(hr))
    {
        errqstr = "ERROR UIAutomation init failed";
        qWarning() << "Get Window Domain:  Cannot initialize UIAutomation.";
        if (uiauto != NULL) uiauto->Release();

        CoUninitialize();
        return;
}

// Convert handle to UIAutomation Element to access address bar.
IUIAutomationElement *windElem;
hr = uiauto->ElementFromHandle(topWindow, &windElem);

if(FAILED(hr))
{
    errqstr = "ERROR Cannot access Window in focus";
    qWarning() << "Get Window Domain:  Cannot access Window in focus.";

    if (windElem != NULL) windElem->Release();
    uiauto->Release();
    CoUninitialize();
    return;
}

VARIANT idVar;
IUIAutomationCondition *cond;
IUIAutomationCondition *cond1;
IUIAutomationCondition *cond2;

// First we store identifier in a VARIANT
// This is required for finding the Address Bar Component
// Create condition to access Address Bar

if (xtrytype == 3) //SearchFireFoxCase
{
    idVar.vt = VT_I4;
    idVar.lVal = UIA_EditControlTypeId;
    // Create condition to make sure it's an edit box we're talking about (specially with firefox)
    hr = uiauto->CreatePropertyCondition(UIA_ControlTypePropertyId,
                                         idVar,
                                         &cond);
    //find the top editboxes
    CComPtr<IUIAutomationElementArray> editboxes;
    //if(FAILED(windElem->FindAll(TreeScope_Children, cond, &editboxes)) || !editboxes)
    if(FAILED(windElem->FindAll(TreeScope_Descendants, cond, &editboxes)) || !editboxes) //use Descendants instead of Children
        return;
    int editboxes_count = 0;
    editboxes->get_Length(&editboxes_count);
    domain = QString::number(editboxes_count);
    for(int icnt = 0; icnt < editboxes_count; icnt++)
    {
        //domain = domain + "icnt:" + QString::number(icnt);
        CComPtr<IUIAutomationElement> editbox;
        if(FAILED(editboxes->GetElement(icnt, &editbox)) || !editbox)
            continue;
        VARIANT urlVar;
        if(FAILED(editbox->GetCurrentPropertyValue(UIA_ValueValuePropertyId, &urlVar)))
        {
            continue;
        }
        else
        {
            // convert URL from BSTR to QString
            domain = QString::fromStdWString(std::wstring(
                            urlVar.bstrVal, SysStringLen(urlVar.bstrVal)));
                           icnt = editboxes_count;  //terminate for loop
        }
    }
    // Cleanup
    cond->Release();
    windElem->Release();
    uiauto->Release(); // Cleanup code for UIAutomation
    CoUninitialize(); // Uninitialize COM
    return;
}

}

PineApple
  • 11
  • 3
  • You mentioned "similar" to a base example. What did you change compared to this. Best would be to provide a [minimal reproducible example](https://stackoverflow.com/help/minimal-reproducible-example) which. – Rene Oschmann Jan 15 '21 at 15:47
  • Added code section – PineApple Jan 15 '21 at 17:40

1 Answers1

1

It seems that if the matching of the address bar string (i.e "Search with Google or enter address") fails, then the continued search of controls within the HTML can be lengthy (depending on the web page). So the solution is to abandon the UI Automation search after several iterations, as supposedly the address bar edit box should be very early within the search. In this case Firefox was using the Bing search engine instead of Google, so the correct address bar string to match should have been "Search with Bing or enter address"

PineApple
  • 11
  • 3