0

I want to send Alt + Mouse Click to some programs.

It works in most programs, it needs a small delay for some, but doesn't work at all in one of them.

The mouse click is working, but the Alt key isn't. If I hold Alt manually and trigger a Mouse Click with SendInput(), it works. So I assume that the Alt key press is not being sent/handled correctly?

This is the code I'm using:

void SendAltClick()
{
    INPUT inputs[4] = {};
    ZeroMemory(&inputs, sizeof(INPUT));

    // Alt down
    inputs[0].type = INPUT_KEYBOARD;
    inputs[0].ki.wVk = VK_MENU;

    // Left down
    inputs[1].type = INPUT_MOUSE;
    inputs[1].mi.dwFlags = MOUSEEVENTF_LEFTDOWN;

    // Left up
    inputs[2].type = INPUT_MOUSE;
    inputs[2].mi.dwFlags = MOUSEEVENTF_LEFTUP;
    
    // Alt up
    inputs[3].type = INPUT_KEYBOARD;
    inputs[3].ki.wVk = VK_MENU;
    inputs[3].ki.dwFlags = KEYEVENTF_KEYUP;

    SendInput(ARRAYSIZE(inputs), inputs, sizeof(INPUT));
}

All calls to SendInput() return 1.

I've also tried Ctrl/Shift + Click, and they work as expected.

Why doesn't it behave the same way with all programs? Why does it need a small delay for some of them? Why doesn't it work at all with one of them?

I can only reproduce with some specific programs. I don't know if I should post the specific details here, since they contain names to commercial products.

EDIT: I went back to the documentation and found the following note at the end of the remarks section, could it be related? How?

An accessibility application can use SendInput to inject keystrokes corresponding to application launch shortcut keys that are handled by the shell. This functionality is not guaranteed to work for other types of applications.

Osama
  • 324
  • 3
  • 11
  • 2
    Calling ZeroMemory() only once is not enough. INPUT is a union, in effect the code signals a mouse down with input.mi.dx = 18, might be large enough to miss whatever you're trying to click. – Hans Passant Aug 25 '22 at 00:26
  • I updated the code. It still doesn't work. Like I said, it works for all of the programs I tested except one, and I didn't see the mouse jumping with the original code. I've also added a note about focus change. Thanks. – Osama Aug 25 '22 at 00:55
  • 1
    Why don't you [SendInput using An array of INPUT structures](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-sendinput#example)? – YangXiaoPo-MSFT Aug 25 '22 at 01:40
  • 1
    [Is it a bug to pass a single-element array to SendInput?](https://stackoverflow.com/q/46744894/1889329) – IInspectable Aug 25 '22 at 07:11
  • @YangXiaoPo-MSFT because it didn't work for all programs. I had to add a 1ms delay in between so it works for some programs. – Osama Aug 25 '22 at 11:02
  • @IInspectable thanks for pointing out that. Unfortunately it doesn't work either way. And it's just better when separated with a 1ms delay. – Osama Aug 25 '22 at 11:49
  • 1
    *"And it's just better when separated with a 1ms delay."* - It is, only if your mental model of input processing doesn't match up with the actual implementation. If you do understand how input is processed by the system, the delay merely turns into an opportunity for bugs. – IInspectable Aug 25 '22 at 11:57
  • @IInspectable I agree that the delay is an opportunity for bugs. I updated my question to say clearly that I **need** a better understanding of how the Alt key press is processed by the system or those programs. – Osama Aug 25 '22 at 12:15
  • Do yourself a favor, get rid of the separate call to `SetCursorPos`, and pass the coordinates inside `input.mi` with the `MOUSEEVENTF_ABSOLUTE` flag set. – Ben Voigt Aug 26 '22 at 14:52
  • @BenVoigt I've removed it since it's off topic anyway. – Osama Aug 26 '22 at 16:32
  • @Osama: it was closely related -- the fact that you want to generate a click at a calculated coordinate instead of wherever the user left the mouse pointer. And the integrated solution, where the click coordinates are in the MOUSEEVENT structures, is superior to the separated method. – Ben Voigt Aug 26 '22 at 17:24
  • @BenVoigt it's not: **If I hold Alt manually and trigger a mouse click with `SendInput()`, it works.** – Osama Aug 26 '22 at 17:35
  • @Osama: I didn't say mean the separated calls will never work. But sometimes they will fail. The integrated method has no race condition. That's why it is better. But it doesn't solve your specific problem. – Ben Voigt Aug 26 '22 at 19:44
  • @IInspectable: If you understand the actual implementation of input processing, then you understand how third-party applications can screw it up. If there's a race condition in the receiving program, then removing the delay/passing all input actions in a single array won't fix that. See my answer. – Ben Voigt Aug 26 '22 at 19:50
  • @BenVoigt Net result: Now two programs are broken. – IInspectable Aug 26 '22 at 19:58
  • 1
    @IInspectable: One program is broken, and one has a failure-prone workaround for the brokenness of the first. But any other workaround would have other drawbacks (hot-patching the bug away? Only works with a single version of a single target program) – Ben Voigt Aug 26 '22 at 20:30

1 Answers1

0

It works in most programs, it needs a small delay for some, but doesn't work at all in one of them.

"Most programs" check the Alt key state from inside the mouse message handler with correct synchronization. There are two correct ways:

"Needs a small delay for some" -- these programs check the Alt key state unsynchronized with the message delivery, probably with

  • GetAsyncKeyState()

but it could also mean the mouse message handler sets a flag and then some later code calls GetKeyState()

"doesn't work at all in one of them"

This program doesn't use the OS input queue (and its abstraction of keyboard state) at all for the behavior you are after, it reads the Alt key from the true keyboard, for example using Raw Input, DirectInput or the HID API. Usually you only see these techniques in games.

To make this "reads the true keyboard" program behave as if the Alt key is pressed, you'll need a USB keyboard simulator/driver sending real HID profile USB events, where the Raw Input portion of the OS will see them. And the program will still be able to detect a difference between "Alt key on keyboard #1" vs "Alt key on keyboard #2"

Ben Voigt
  • 277,958
  • 43
  • 419
  • 720
  • Thanks for the good explanation. But I noticed that when I send an Alt click, the window menu gets focus, doesn't that mean that it's not using raw input? – Osama Aug 26 '22 at 20:11
  • @Osama: `DefWindowProc`, which is responsible for opening menu, doesn't use raw input. In fact that's exactly where you would expect SendInput key events to go, bypassing the application-specific processing, if the application-specific logic is built on Raw Input/DirectInput/HID API. – Ben Voigt Aug 26 '22 at 20:26
  • Though that does make my claim "not used AT ALL" incorrect, so I've edited that part. – Ben Voigt Aug 26 '22 at 20:27
  • Ok this makes sense now. Final question: is it possible to know which API the program uses and maybe try to simulate it (e.g. send a WM_INPUT)? – Osama Aug 26 '22 at 20:33
  • You can use something like http://www.rohitab.com/apimonitor to discover what API the program is using. I already gave some guidance on synthesizing raw input -- a real driver that doesn't connect to real hardware. Microsoft provides a driver example `vhidmini2` that looks pretty close "The sample also is useful in testing the correctness of a HID report descriptor without using a physical device". You might also be able to use the Remote Desktop Connection feature of the OS in some way (since it includes input drivers that receive their input from the network then synthesize HID events) – Ben Voigt Aug 26 '22 at 20:39
  • @Osama: Before digging into synthesis, I would test making a Remote Desktop Connection in the normal way, and see if doing Alt+Click over the remote connection affects the application in the way you desire. – Ben Voigt Aug 26 '22 at 20:40