Is there any way to fully control browser from application?

Question

I was trying to make a pc application that helps people more conveniently when they are using a browser(e.g Chrome, Firefox, Edge ...)

For example, the application shows the default browser and let user can choose another browser as default if they want to, and the app shows all windows that are currently opened; Overmore, when the user clicks a specific window then app will focus the window.

^{user can select default browser}

^{user can see all windows categorized by browser type}

So, my initial planning was using Election js; however, I've found that there is no way to control a browser from the Electron application since Node js cannot access the users' system(this thinking could be wrong since I have poor English)

After some research, I'm guessing C# can do that by using windows api. I've saw this post that saying by using DDE, we can get tabs urls.

But I'm wondering if C# can fully control a browser; for instance, create a new window, notice if music or video is playing in the tab, request to browser for getting favorites list.

If it cannot, how about requesting api to the browser from pc application?

Answer 1

You cannot fully control a browser. Internet Explorer was the only browser that had a documented and supported API to control, inspect and automate. Internet Explorer is dead so this is a dead end. The basic DDE control interface is also outdated.

Accessing favorites and the content of open tabs is limited because evil people would use such APIs to inject and spy. If you want to access the favorites you will have to write custom code for each browser.

Using the accessibility and UI automation APIs is the only reliable and supported way of interacting with modern browsers. SetWinEventHook can be used to detect window creation and primitive state changes. MSAA/UI Automation needs to be used to get more information from each browser window.

The thumbnail API and/or magnifier API can be used to get the preview image.

To open a new tab, execute a new process with the URL on the command line. Some per-browser customization might be required (-new-tab parameter etc).

Use IApplicationAssociationRegistration::QueryCurrentDefault to detect the default browser.