WPF Stylus Events when Stylus/Pen is away from the screen

Question

Is it possible to react (via StylusButtonDown etc., or alternatives) to button presses (i.e. one of the stylus buttons, not buttons in some app) that happen when the stylus/pen is not touching the screen, i.e. hovering in the air somewhere in range of the notebook? Those only seem to trigger if the pen tip is actually touching the surface of the screen. Specifically, I do not need to know about the position of the button. Just literally when the barrel button is pressed.

I'm using a Microsoft Surface and the Surface Pen that comes with it in particular, if that makes any difference. I don't need it to be cross-platform, portable or anything. In fact, solutions in other languages (C++, etc.) are OK. Hacky solutions are very welcome.

score 3 · Answer 1 · answered Feb 01 '20 at 20:03

I have might have solution that would work with currently used technologies such as bluetooth and Wifi that is in pen and your Microsoft Surface. But you will need to program some parts on your own (most of mentioned techniques are in open source libraries).

After you pair your,pen with Surface you should be connected with Wifi and bluetooth to the pen.

1. Option

There is good article, explaining bluetooth triangulation that we will use for triangulating pen in close area, except we will use bluetooth and wifi (we dont have 3th point) so it will not be so precise as 3 points. But with 2 points you can actually measure distance by this table and find where is theirs intersection, in such small space its doable.

Use similar method with Wifi for second triangulation point, you can see source codes for Wifi triangluation down there.

Next step will be calibrating the position of pen in holder, that means that the pen will be at one side of the Surface(for simplification lets assume its only at right shorter side)

from this you can compute area of screen

Allowed area:

offsetHeightOfPenScreenRation = ScreenHeight - penHardwareHeight

NonValidX = StartpenPositionX - ScreenWidth < penPositionX or StartpenPositionX < penPositionX

NonValidY = StartpenPositionY - ScreenHeight < penPositionY or StartpenPosition+offsetHeightOfPenScreenRation < penPositionY

penIsClose = not ( NonValidX or NonValidY)

This is how it could work, you will need to keep this software running in background and it could have some serious impact on battery, you will need external wifi adapter for internet.

I dont have Microsoft surface so i cant code it myself cause i dont have device to try it on, but this is working idea. With little tuning of precision it could be really precise positioning.

2. Option

Wifi triangluation: https://github.com/kevindu/wifi-ap-positioning can be maybe used alone and you can transfer location through the bluetooth if you can get into a firmware of pen, but this option will not be so precise.

Gopinath · Accepted Answer · 2020-02-05T01:51:46.870

AutoHotKey is the answer!

AutoHotKey is an open source tool that helps create custom automated responses to Windows input device events.

AutoHotKey supports Surface Pen. It also supports the scenario of pen being away from screen.

Surface Pen connects to Windows via Bluetooth, with a signal range of several metres. Therefore, it is technically possible to use a Surface pen as a remote controller for Windows!

One of the most popular scenarios is Using Surface Pen as a Presentation controller

Here is a working AutoHotKey script for controlling Powerpoint presentation with Surface Pen:

;  File name: surface-pen-slide-show-control.ahk
;
;  This program helps the presenter scroll through slides, using Surface Pen
;  
;  Single-click on pen button to scroll forward
;  Double-click on pen button to scroll backward
;

#NoEnv
SendMode Input
SetWorkingDir %A_ScriptDir%

#IfWinActive ahk_class screenClass
#F20::
  Send, {Right}
Return

#F19::
  Send, {Left}
Return

More information and Help:

The Complete Reference on AutoHotKey: https://www.autohotkey.com/
Demo code (pen away from screen): https://github.com/jonathanyip/Surface-Pro-3-AutoHotkey/blob/master/Pen.ah
Remote Control for Powerpoint slide show: https://anderseideblog.wordpress.com/2015/08/04/turing-my-surface-pen-into-a-presenter-tool/

Thanks, that sounds promising! I'll try it out to see if it works. — TravisG, Feb 05 '20 at 09:50

score 0 · Answer 3 · answered Jan 30 '20 at 14:44

As far as I know, Microsoft Surface does not have sensors around the screen. So no, there are no events that would react to stylus/pen when they are near the screen.

Although, if you are an engineer, you might create some sensors yourself, and then create events that would react to such things.

score 0 · Answer 4 · answered Feb 04 '20 at 04:13

The only way I see this is to incorporate the very sensors for the buttons yourself and since it is already connected using bluetooth or WiFi, so use the required triangulation method to calibrate the device. Well, a hacky answer I believe, since Microsoft Surface does not have sensors around the screen.

score 0 · Answer 5 · answered Feb 04 '20 at 07:33

If you are really looking for a "hacky solution", then I have an idea, but it will be a really dirty way to do it. Despite the fact that no api supporting the button exists, the system responds to pressing the button even if the stylus does not touch the screen. You can create an application that reads the following command line arguments at startup: ClickOnce, DoubleClick, PressAndHold. Depending on the argument provided, the application should send the appropriate message to your main application. Now you can go to the "pen settings" and depending on the action, select to run the above application with the appropriate parameter.

WPF Stylus Events when Stylus/Pen is away from the screen

5 Answers5