2

I am setting up an automation framework used to automate certain flows in a JavaFx Application. There is a new use case where we need to display an external third party web-page within our application using SWT Browser API.

enter image description here

Since, my application uses JavaFX and not SWT, I had to embed the swt browser using the approach mentioned here. We had to use swt browser because the API provided by the third-party application uses swt event listeners to communicate with my application.

Now that the browser is embedded, we need to test certain data-exchange scenarios using automation tests. I have tried to automate the flows using the SWTBrowser.execute(String script) method by writing some wrapper methods. Sample code:

    public void setTextById(String id, String text) throws InterruptedException
    {
        Browser browser = getSwtBrowser();
        CountDownLatch latch = new CountDownLatch(1);
        browser.getDisplay().syncExec(new Runnable()
        {
            @Override public void run()
            {
                boolean result = browser
                        .execute("document.getElementById('" + id + "').setAttribute('value', '" + text + "');");
                latch.countDown();
            }
        });
        latch.await();
    }

But this approach seems to be reinventing the wheel and prone to a lot of bugs. I plan to use Selenium to do some content access/manipulation in the web page as we are already using Selenium for other use cases in the application.

The SWT Browser uses the native browser of the OS it is deployed on to render the web pages as suggested here. It is using:

  • IE 11 on Windows
  • Safari 11 on Ubuntu

Question: How could we get Selenium to connect to the WebDrivers of the SWTBrowser's embedded OS browsers to access the dom in the existing browser session?

EDIT: Did a little more debugging and noted the following points:

Selenium has two parts for automation testing. The client and the server. The client sends the automation operations as a HTTP POST request to the server. The server is responsible for spawning the desired browser and perform the actual interaction.

I was going through the server's code and found out that the server launches the native browser using either IELaunchURL()/CreateProcess() api. Also there is an attachToBrowser(...) method.

I would like to use the attachToBrowser(...) method by sending the process information of the Internet Explorer which is embedded in the SWTBrowser. There is also another difficulty in this approach. It looks like the SWTBrowser doesn't spawn the IE process directly. It interacts with the dlls of the internet explorer. resources used by the SWT browser application Is there anyway I can extract the process information of the embedded IE process in this case? Or Is there any other approach I can go by to solve this?

Jayan
  • 18,003
  • 15
  • 89
  • 143
Abhijith Gururaj
  • 437
  • 4
  • 14

1 Answers1

1

You would have to start a new IE WebDriver and then attach it to the existing browser. I dig around a little and all I could come up with is this old thread: Can Selenium interact with an existing browser session?

This approach is using the RemoteWebDriver so you will need the standalone version of Selenium. For me - it seems like an overkill, probably prone to errors. What I would suggest is to integrate some sort of image recognition software that will interact with embedded browser. I know Sikuli does something like this and it has a Java API available. You can get it here: Sikuli

Moro
  • 781
  • 4
  • 14