3

How can I send keystrokes and mouse movements to a specific running program through its PID. I've used both pywinauto and pynput, and they work great, but I want to send keys to a program that is not in focus. I found this question: How to I send keystroke to Linux process in Python by PID? but it never explains what filePath is a path to.

If you could help solve for this example, that would be great! I want to send the "d" key to an open Minecraft tab for 10 seconds, and then send the "a" key for the next 10 seconds and stop. I would need this to be able to run in the background, so it could not send the keys to the computer as a whole, but only to the Minecraft tab. I am on Windows 10 by the way.

Any help would be appreciated!

Timmy Diehl
  • 409
  • 3
  • 16
  • Just for context, why do you want to send keystrokes and mouse movement to a window that is not in focus? I'm just trying to better understand your use case. – Shane Bishop Apr 01 '22 at 20:22
  • @ShaneBishop Using the Minecraft example, I would want to have my character run forward for 20 minutes or something along those lines, but I would like to be doing other things on my computer at the same time. I also might want to have my character turn 90 degrees every 10 minutes, so I would need mouse movement as well – Timmy Diehl Apr 01 '22 at 20:25
  • 7
    I feel this question might be too broad for Stack Overflow. Maybe you could rephrase your question to something more specific, like "How to emulate an Up Arrow press and hold for x minutes for a specific Windows process, using Python?" – Shane Bishop Apr 01 '22 at 22:53

2 Answers2

2

With ahk you can do this with Python+AutoHotkey

pip install ahk
pip install "ahk[binary]"

from ahk import AHK
from ahk.window import Window
ahk = AHK()
win = Window.from_pid(ahk, pid='20366')
win.send('abc') # send keys directly to the window

Note that some programs may simply ignore inputs when they are not in focus. However, you can test this works in general even when not in focus by testing with a program like notepad

Full disclosure: I author the ahk library.

sytech
  • 29,298
  • 3
  • 45
  • 86
  • Wow! This library is incredible!! Quick question though, how can I hold down the mouse button while running other keys? – Timmy Diehl Apr 08 '22 at 14:35
  • @TimmyDiehl a typical mouse click has two events: "DOWN" and "UP". You can send these events independently with the `click` method by passing the `direction` keyword argument. For example, to hold down the mouse, type something, then release the mouse: `ahk.click(direction='down');ahk.type('some words');ahk.click(direction='up')` – sytech Apr 08 '22 at 18:39
1

Pretty sure you won't be able to, at least not easily let me explain a little bit how all of this works.

Lets start with the hardware and os, the OS has certain functions to read the input you give the computer. This input goes into a "pipe", the OS is reading input, and putting into the pipe, on the other side of the pipe there may be an application running, or it may not. The OS typically manages this (which app to put on the pipe listening) by defining which app/window is active. Apps access this pipe with the API given by the OS, they read the input and decide on it.

The libraries you cited above, change the values of the keyboard and mouse, in other words, they make the OS read other values, not the real ones, then the OS puts them in the "pipe", and are read by the app that is listening on the pipe (the one active). Some apps have their own API's for this, but I would guess Minecraft doesn't. If they don't have an API, what can you do? well, as I said, nothing easy, first of all "hacking" the app, in other words change it to listen to some other input/output rather than the one given by the OS, (this would be you making your own API). The other one would be you changing the OS, which would also be extremely hard, but maybe a tiny bitty easier. It also depends on your OS, I think Microsoft does offer input injection api's

So, simple options, first, run a VM with a GUI and use pywinauto, pyautogui, etc. The other option would be if you can run it in the browser, do so, and use something like Selenium to automate the input.

Quick note, why does selenium works and the browser can read input in the background? Easy, it's not, it just executes the code it would execute if it would have read the input! javascript, cool isn't

Raul Jerlach
  • 167
  • 2