chrome.debugger.sendCommand() Input.dispatchMouseEvent error on MV3

Question

I'm struggling with this for hours now..

I'm currently writing a Chrome Extensions, its goal is to automate click on a website. Since the website is checking for the isTrusted property, I have to emulate the click from chrome.debugger (Or at least, that the only way I found).

I have actually no one, but two problems.

1st one: If I set opts.x / opts.y "dynamically", it results in this error: Uncaught (in promise) Error: {"code":-32602,"data":"Failed to deserialize params.x - BINDINGS: mandatory field missing at position 55","message":"Invalid parameters"}

2nd one: Tried putting the value directly, it's working, but it is not clicking on the provided coordinates, it's actually clicking lower.

This is my code:

background.js

chrome.runtime.onMessage.addListener(function(msg, sender, sendResponse) {
    if (msg.text == "click that button please") {
        let infos = []
        chrome.debugger.attach({tabId: sender.tab.id}, "1.2", function() {
            let clicked  = false
            
            let x = Math.floor(msg.button.offsetLeft + msg.button.offsetWidth / 2)
            let y = Math.floor(msg.button.offsetTop + msg.button.offsetHeight / 2)

            opts = {type: "mousePressed", button: "left", x: x, y: y, clickCount: 1}

            chrome.debugger.sendCommand({tabId: sender.tab.id}, "Input.dispatchMouseEvent", opts)
                .then((e) => {
                    infos.push(e)
                });
    
            opts.type = "mouseReleased"
   
            chrome.debugger.sendCommand({tabId: sender.tab.id}, "Input.dispatchMouseEvent", opts)
               .then((e) => {
                   infos.push(e)
               });
        })
        sendResponse({clicked: infos});
    } else {
        sendResponse({error: 404});
    }
    return true;
});

content.js


var buttons = {
    "market": "#app > div > div.mt-3 > div.d-flex.justify-content-between.align-items-baseline > div:nth-child(2) > button"
    }
}
!async function() {
    window.onclick = function(e) {
        console.log(e.target, e)
    }

    let target = document.querySelector(buttons['market']); // Working
    console.log(target);

    chrome.runtime.sendMessage({text: "click that button please", button: target})
        .then((result) => {
            console.log('Did it clicked?', result); // result = []
        });
}();

manifest.json

{
    "name": "xxx",
    "description": "xxx",
    "version": "0.0.1",
    "manifest_version": 3,
    "author": "Me",

    "host_permissions": [
        "https://*.xxx.xxx/*"
    ],
    "permissions": [
        "debugger",
        "activeTab",
        "tabs",
        "scripting"
    ],
    "background": {
        "service_worker": "background.js"
    },
    "content_scripts": [{
        "matches": ["https://*.xxx.xxx/*"],
        "js": ["content.js"]
    }]
}

wOxxOm · Accepted Answer · 2023-01-22T19:08:35.190

1

You can't send DOM elements via messages. It's not serializable, so an empty object arrives and your formula results in a NaN, i.e. not a number.

Solution: send an object/array with the element's coordinates.

Your use of offsetXXX props is incorrect, AFAICT, as the documentation for dispatchMouseEvent says x/y are relative to the viewport.

Solution: getBoundingClientRect().

const bb = elem.getBoundingClientRect();
chrome.runtime.sendMessage({x: bb.left, y: bb.top});

P.S. An alternative solution is to replace EventTarget.prototype.addEventListener or onclick setter of Document.prototype in the MAIN world of the page, so that your listener will call the site's listener passing a Proxy(event, handler) where handler returns true for isTrusted.

edited Jan 22 '23 at 19:08

answered Jan 22 '23 at 18:58

wOxxOm

65,848
11
132
136

Oh man I love you. Working perfectly now with both of your solutions. Many thanks! – PyG Jan 22 '23 at 19:58
I still have some issues making it click on the good element.. I am now calculating the center of the element with your code snippet and script is clicking there. But sometimes the click target is somewhere near the button, beside, on top, lower.. – PyG Jan 22 '23 at 20:10
1

@PyG, maybe it's caused by the debugger warning changing the viewport height? One workaround would be to call chrome.debugger.attach first, then getBoundingClientRect. Another solution would be to verify the coordinate in the content script using `document.elementsFromPoint` and adjust the position by 1 until it is found. – wOxxOm Jan 22 '23 at 22:25
Attaching before `getBoundingClienRect` was the solution (Tried to implement `elementsFromPoint`, but my current architecture was preventing it). Thanks again – PyG Jan 23 '23 at 08:48

chrome.debugger.sendCommand() Input.dispatchMouseEvent error on MV3

1 Answers1