2

I am looking for a way to script mouse clicks and key presses, and pipe them to a particular program without interfering with the mouse and keyboard output to other programs the way a utility like autopy or PyUserInput do. I want only one particular program to "think" that the mouse is being moved in a particular way, or certain keys are being pressed.

A python solution that works in Linux would be best.

clj
  • 303
  • 2
  • 5
  • 13
  • While you can send fake keyboard and mouse events, you cannot fake keyboard and mouse *state*, which a program is free to query. Your best bet is to run your program in a dedicated X server like Xnest and manipulate its pointer and keyboard devices. – n. m. could be an AI Oct 14 '15 at 06:38

1 Answers1

0

Not an easy task, primarily because the keyboard and mouse event actually go to the X server and the window manager which then passes them to the appropriate application depending on the number, screen position and state of the app's windows.

Generating keyboard and mouse events is possible (see this Q&A: How can I control the keyboard and mouse with Python?) however these events would normally interfere with the regular/main X server, window manager and the other apps, which is something you don't want.

The solution is a program which acts like an X Server + window manager for the app to be controlled. Easiest approach is indeed along the lines that @n.m. suggested - a dedicated X server in which you run only the app you want to control and your controlling app. Such solution does not interfere with the main X server, its display manager and its applications. VNC is another popular such server you could use.

Your controlling app would need to locate the windows that belong to the app to be controlled, their position and state and adjust the generated keyboard and mouse events accordingly (i.e. play a portion of the window manager role, or at least be able to understand and follow some of the states and actions of the real window manager). For example the window manager might not always open a window of the app to be controlled in the same screen position, so your controlling app would need to adjust the position of a mouse click event (which uses X screen coordinates) depending on where the app's window is positioned on the screen.

Alternatively you might be able to first control the app's window and bring it into a well known state (say window visible, of certains size and in a certain position, on top of other windows, selected/with focus on it) such that the controlling sequence of mouse and keyboard events lands in the expected places.

Community
  • 1
  • 1
Dan Cornilescu
  • 39,470
  • 12
  • 57
  • 97