2

I am trying to find a way to simulate/automate mouse motion using the Wayland protocol on a debian based OS as did using Xlib in X11/X Window System giving the x and y coordinates:

from Xlib.display import Display
from Xlib.ext.xtest import fake_input
from Xlib import X
import os

#Go to x and y coordinates and click with left button (1):
x = 26
y = 665
button = 1

myDisplay = Display(os.environ['DISPLAY'])

fake_input(myDisplay, X.MotionNotify, x=x, y=y)
myDisplay.sync()
fake_input(myDisplay, X.ButtonPress, button)
myDisplay.sync()
fake_input(myDisplay, X.ButtonRelease, button)
myDisplay.sync()

and with pynput:

from pynput.mouse import Button, Controller

mouse = Controller()

mouse.position = (26,665)

mouse.click(Button.left)

I have tried with uinput to reproduce the same idea from the scripts presented above, the code doesn't give error but nothing happens when the script is executed:

sudo modprobe uinput && sudo python3 uinputtest.py

import uinput

# Create new mouse device
device = uinput.Device([
    uinput.BTN_LEFT,
    uinput.BTN_RIGHT,
    uinput.REL_X,
    uinput.REL_Y,
])

# Move the pointer to x = 26 and y = 665
device.emit(uinput.REL_X, 26)
device.emit(uinput.REL_Y, 665)

# Click and release the left mouse button 
device.emit(uinput.BTN_LEFT, 1)
device.emit(uinput.BTN_LEFT, 0)

Is there anything missing on this last script? I am trying to execute it on Wayland with super user permission.

theEarlyRiser
  • 134
  • 12
  • 1
    probably depends on your Wayland compositor, since there is no such thing as "on Wayland", but only specific compositors – user253751 Feb 15 '23 at 13:19

1 Answers1

2

You should wait a bit before emitting any event.

In uinput example code it says:

We are inserting a pause here so that userspace has time to detect, initialize the new device, and can start listening to the event, otherwise it will not notice the event we are about to send. This pause is only needed in our example code!

Putting a time.sleep(1) after the device intialization fixes the problem. Ydotool, which provides a userspace application and daemon for what you are trying to achieve also waits for a second.

Kljunas2
  • 56
  • 1
  • 6
  • It worked here very well thanks for the tip! Only the mouse positions that I realized now that when I do ```device.emit(uinput.REL_X, 26)``` > ```device.emit(uinput.REL_Y, 665)``` the mouse pointer goes x and y units from the current position which my mouse is located, I am searching here how I do to go without need to use the current position to x and y like in this snippet from the Xlib code: ```fake_input(Display(os.environ['DISPLAY']), X.MotionNotify, x=26, y=665)``` > ```myDisplay.sync()``` – theEarlyRiser Feb 17 '23 at 18:02
  • Instead of `REL_Y` and `X`, you can use `ABS_Y`. – Kljunas2 Feb 17 '23 at 18:45
  • I have tried with ```ABS_X``` and ```ABS_Y``` now but the mouse doesn't click and move anywhere, I am gonna give a check if it has anything to do with the device – theEarlyRiser Feb 17 '23 at 19:05
  • I'm trying to use these ABS_X and ABS_Y attributes but no effect yet, I saw some codes around the net where the people use these attributes to move joystick for example using: ``` ...device = uinput.Device([... ...uinput.ABS_X + (0, 255, 0, 0), uinput.ABS_Y + (0, 255, 0, 0),...``` ... ]) ``` What this tuple means (v1, v2, v3, v4) when concatenated with ```ABS_X``` and ```ABS_Y```?! I was able to let my mouse static at 0,0 (x, y) coordinates and the cursor went to 26, 665 (x, y) when I let the code with REL_X, REL_Y and when I put ABS_X and ABS_Y in device creation nothing happened – theEarlyRiser Feb 22 '23 at 18:50
  • It seems that the project is quite old (last commit 7 years ago). Some things may have changed in udev library and can no longer work. You can check the source code on Github and compare it with some other project like ydotool, which does work for me. – Kljunas2 Feb 22 '23 at 20:32
  • Thanks for the tip again @Kljunas2, I have searched in the whole net and I asked today in the python-uinput github and I am waiting any answer from them, I have tested python-evdev but this one is hard to inject mouse events compared with uinput... About ydotool I have installed here using ```sudo apt install ydotool``` but it isn't working here i'm getting ```ydotool: notice: ydotoold backend unavailable (may have latency+delay issues)``` and the mouse pointer doesn't do nothing, I am gonna give another check on it – theEarlyRiser Feb 23 '23 at 07:21
  • Have you run the daemon first? `sudo ydotoold` You may also have to specify some other socket location and permissions. – Kljunas2 Feb 23 '23 at 08:46
  • Hi @Kljunas, sorry for the late reply, yes! I was able to install and execute ydotool last week, I have realized that the ydotool version installed by ```sudo apt install ydotool``` is the old one, I have build the latest cloning their repository on github and after building the content through cmake, I never used cmake before and they let the ydotool doc very incomplete at their official github. – theEarlyRiser Mar 07 '23 at 19:31