How to automate UI keyboard input? pywin32 PostMessage vs Powershell SendKeys

Question

Motivation: I'm using a software that doesn't have an API to interface with ... I have no alternative and have to open the software, send simple key sequence, then close ... Again and again, so I want to automate this process.

Goal: Send combinations of keyboard inputs to an inactive window.

Progress: I wrote a powershell script that open, send keys, wait, then end the process, but it only works on active window. A part of the powershell code is as follows.

$appProcess = Start-Process -FilePath $path -PassThru 
$wshell = New-Object -ComObject wscript.shell;
$wshell.AppActivate($appProcess.Id)
$wshell.SendKeys('%(E)E')
Stop-Process $appProcess -Force

It works, but only on active window (windows comes to the top). What I want is to run automate the window in the background. I found an article that point me to using PostMessage in Win32 API. Since the majority of my code uses python, I decided to move from powershell to pywin32.

Issue: I cannot get the PostMessage to send key to the right handler. I saw in this article that I may need to find the exact window, but I still don't really understand how. In powershell, I can directly send keys via $wshell.AppActivate($appProcess.Id).

hwndMain = win32gui.FindWindow(None, winname)
hwndChild = win32gui.GetWindow(hwndMain, win32con.GW_CHILD)
temp = win32api.PostMessage(hwndChild, win32con.WM_KEYDOWN, 0x45, 0)
# temp came out as None

Question: Is there a way to do this in pywin32 / Win32 API?

Edit: (May 8, 2020) Yes, I have heard that using SendKeys are not reliable, but since there is no alternative offered to questions like this one on SO, how should anyone learn the "right way"? If you think there is alternative, everyone will appreciate to see a solution in action. Please suggest edits to my post to improve the quality of the question instead of shooting it down.

See https://winsourcecode.blogspot.com/2019/05/winlistexe-list-open-windows-and-their.html to list all windows. An easier but less reliable way is shown here https://winsourcecode.blogspot.com/2020/01/getwindowrectexe-reports-on-windows.html. It is about windows not processes. You need to send the resulting message not keys. — , May 06 '20 at 19:47
MS-DOS doesn't exist any more. It hasn't existed in more than a decade. The Windows terminal (command) window is NOT MS-DOS. — Ken White, May 06 '20 at 19:53
@KenWhite Thanks for the note. I edited the post to reflect that. — techtana, May 06 '20 at 20:02
"*I found an article that point me to using PostMessage in Win32 API*" - [You can't simulate keyboard input with PostMessage](https://devblogs.microsoft.com/oldnewthing/20050530-11/?p=35513) — Remy Lebeau, May 06 '20 at 20:56
Why does this question get downvoted? I believe I framed my question clearly, and Don't see any explanation for the downvote. — techtana, May 08 '20 at 19:05

score 0 · Answer 1 · answered May 06 '20 at 23:24

0

Time ago I developed SendMessage utility that allows to post messages to other processes or Windows via the WIN32 API. I discovered that I can't send a message to an inactive window, so I searched for a method to activate a Window. After several tests, I found a trick that allows me to "reactivate" a Window:

The development of this program presented a problem: the items in the Notepad topmost menu can not be selected if the window was not active. I did some tests and discovered that this point depends on the relation that the cmd.exe window have (has?) with the other window and the particular program in the other window. For example, in the case of Calculator Windows accessory, its menu items can be selected when the window is not active, and even when the window is minimized! I tried to develop a method to activate other window from the cmd.exe one using just System-Defined messages. I did several tests using diverse combinations of WM_ACTIVATE, WM_CANCELMODE, WM_ENABLE, WM_ACTIVATEAPP, WM_SETFOCUS and WM_KILLFOCUS messages, with no success. Fortunately, the items in the system menu of any window can be selected from another window, and after a SC_RESTORE the restored window remains active; this behavior makes possible to activate Notepad and other windows from the cmd.exe one via a Minimize/Restore procedure.

I think that using my SendMessage.exe utility you may test if you may send keys to your inactive window process or to activate it, so you may then translate such a method to your phyton code. For complete details on this matter, see this link.

answered May 06 '20 at 23:24

Aacini

65,180
12
72
108

There's a specific tool that allows you to automate a UI without resorting to ungodly hacks. It goes by the name [UI Automation](https://learn.microsoft.com/en-us/windows/win32/winauto/entry-uiauto-win32). UI Automation doesn't impose a whole lot of requirements. For example, UI Automation doesn't require the target to be active, in the foreground, or even be visible. – IInspectable May 08 '20 at 07:38
@IInspectable: I think you ignored this line in the question: _"**Question:** Is there a way to do this in pywin32 / Win32 API?"_ Using the elements provided in my answer the OP can, in a very simple way, test the sequence of Win32 API calls that solve the problem (either sending keys to an inactive window or reactivate an inactive window), so I think my answer answers the question as posted. IMHO your _recommendation_ about using UI Automation tool should be directed to the OP instead, not to me... Deserves this situation a downvote? Really? **`:/`** – Aacini May 08 '20 at 12:35
The vote count on a proposed answer is useful to future visitors looking for a solution. Both of your approaches fail to deliver. Sending input messages cannot be used to reliably automate a UI. And suggesting to change the preconditions by forcing a window active does not address the question on how to automate a UI that's not active. That's ignoring that activating a window isn't even enough. It would have to become the foreground window, too, to be able to receive input using the supported way to inject input. – IInspectable May 08 '20 at 12:45
1

@IInspectable Thanks for your comment ... I have not even heard of UI Automation API before! ... I know I am novice. I have been searching through SO for solutions similar to my case, but I have not seen any of them mentioned this API before. So I think with more examples and explanation, you could have provided a solution to my question. Yes, I have heard many times that using SendMessage are not reliable, but since there is no alternative offered to questions such as mine, how should anyone learn the "right way"? I appreciate your comment and would love to see solution in action ... – techtana May 08 '20 at 18:57
@techtana It IS NOT unreliable, it just has rules. – May 09 '20 at 02:12
@mar Indeed. One such rules is that [replaying input is not the same as reprocessing it](https://devblogs.microsoft.com/oldnewthing/20121206-00/?p=5903). Which makes faking input by sending messages unreliable. This [has been noted](https://stackoverflow.com/questions/61643934/how-to-automate-ui-keyboard-input-pywin32-postmessage-vs-powershell-sendkeys/61644350#comment109042298_61643934) previously. – IInspectable May 17 '20 at 16:16
@IInspectable MY ANSWER DOES NOT FAKE INPUT. It sends commands which are commands not input by sending WM_Command. It is common for programs to send themselves a WM_Command. EG Choosing Save from the file menu sends a WM_Command for Save, if it is a new file the save procedure will send a WM_Command for SaveAs. My program shows controlling Notepad WITH NO USER INPUT. – May 17 '20 at 21:00
@mar Why did you assume I was commenting on *your* answer? If I needed to comment on your answer, I would leave a comment on your answer. – IInspectable May 17 '20 at 21:32
Everyone else on this page is doing nothing about user input EXCEPT you. Everyone is sending messages which aren't input. – May 17 '20 at 21:36
@mar *"you may test if you may send keys to your inactive window"* - Maybe it's not just me after all. Also, if you are having difficulty keeping your temper under control, Stack Overflow is [the wrong place](https://stackoverflow.com/conduct) to demonstrate this. – IInspectable May 18 '20 at 07:27

How to automate UI keyboard input? pywin32 PostMessage vs Powershell SendKeys

1 Answers1