Include mouse cursor in screen capture using SharpDX

Question

I'm using the ScreenCapture sample of SharpDX library to be able take a screenshot of a game that is in fullscreen (because common GDI+ capture techniques here doesn't work, even the Windos default "Print" key cannot take a proper screenshot of the game).

The code that I'm using in my application is practically the same of the official sample linked above (translated to VB.NET), so I think is not necessary to publish the same code block here.

The problem I have is that I would like to include the mouse cursor in the captured image.

Maybe SharpDX has some method or property/parameter to include the mouse cursor in a friendly way?, if not then how could acchieve this taking into account that the mouse cursor of the game is personalized (not the default white Windows arrow)?.

Have you tried getting the graphics from the resulting image and calling [**Cursor.Draw()**](https://msdn.microsoft.com/en-us/library/system.windows.forms.cursor.draw(v=vs.110).aspx)? — Visual Vincent, Jul 02 '16 at 22:03
@Visual Vincent I do exactlly that when doing a common capture with a Bitmap and `Graphics.CopyFromScreen()`, however this seems more complicated because its a game which its native resolution is smaller than the screen resolution and it is running in fullscreen mode (so it reescales the resolution, and no way with .NET neither windows/Print-key can't capture an image of the game with the real screen size in that circunstances, only with DirectX techniques as I discovered), so its complicated for me to locate the exact mouse hotspot in that circunstances. — ElektroStudios, Jul 02 '16 at 23:11
If **SharpDX** can capture an image of the screen, I suppose that it could do something itself to also capture and draw the cursor in the resulting image, avoiding calling the WinForms related namespace members to help on this. Anyways the `Cursor.Draw()` (or `Cursors.Arrow.Draw()`) will draw a default Windows cursor, but I would like to capture the cursor that I see in-game when I do the screenshot. — ElektroStudios, Jul 02 '16 at 23:19
I understod you wanted that, though is `Cursor.Draw()` really equivalent to `Cursor.Arrow.Draw()`? I thought maybe it'd draw the current cursor instead. — Visual Vincent, Jul 02 '16 at 23:37
The resolution of the game shouldn't matter since the screen's resolution will be the same as the game's (if run in fullscreen). You don't need to locate the hotspot yourself because it will be `Cursor.Position`. — Visual Vincent, Jul 02 '16 at 23:50
Hmmm, I tested the mouse position, you have reason. Anyways, what would happen with those certain games that retains the mouse cursor at the center of the screen while you are moving the mouse in-game?. — ElektroStudios, Jul 02 '16 at 23:56
Those draw their own cursor AFAIK, so it should be included in the image. Though I don't know for certain, I'm not much into how these things work... Maybe this'll help getting the cursor image: http://stackoverflow.com/questions/918990/c-sharp-capturing-the-mouse-cursor-image — Visual Vincent, Jul 03 '16 at 00:01
@Visual Vincent Thanks for comment and trying to help. I will read and try what is explained in that linked answer. — ElektroStudios, Jul 03 '16 at 00:03
No problem. Screen recorders seem to use some technique to record the cursor, but I don't know what exactly. Maybe DirectShow has such possibility? -- I also found a simpler way of getting the cursor image/icon (if it works). See the last post, starting at `CURSORINFO cursor`: https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/f85d3741-2396-41fd-b054-ccb3c1298217/mouse-capture-in-directshow-desktop-recording-application?forum=windowsdirectshowdevelopment -- Maybe you could make a simple C++/CLI wrapper for it? — Visual Vincent, Jul 03 '16 at 00:47
It is also explained in [**this CodeProject article**](http://www.codeproject.com/Articles/12850/Capturing-the-Desktop-Screen-with-the-Mouse-Cursor) under "Solution". — Visual Vincent, Jul 03 '16 at 00:51
The problem that I see with all those solutions is that are Win32 based, I mean unmanaged code that maybe in some poor-tested scenarios could give unpredictable results such bad colors in the resulting cursor or a cursor that doesn't corresponds. I really would like to beneffit from this DirectX wrapper to do this, because they handle that for me, which I think (I think) would be the most efficient and faster way to do it. — ElektroStudios, Jul 03 '16 at 01:35
Just to mention it: The class name "Win32stuff" of the CodeProject solution makes me think that is a code copied from the C# answer you linked before (or viceverse,the C# guy copied from that article),its basically the same in the three solutions.However,to be honest I didn't knew that technique to capture the cursor and from what I tested by the moment seems it works as expected,I can capture the mouse cursor that I see in the game, but I still didn't implemented it in the app. to do deeper tests with a screenshoot and see the result,its too late here,tomorrow I will do. Thanks for all again! — ElektroStudios, Jul 03 '16 at 01:36
Glad to hear! So, are there any more missing pieces or is your little puzzle complete? — Visual Vincent, Jul 03 '16 at 20:04
Well, I really would like to discover a DirectX based solution to beneffit from this technology and to avoid all the dirty unmanaged code full of p/invokes to capture the mouse cursor icon, however... as always no body notices my DirectX questions because I never receive a answer with a pure DirectX/SharpDX solution even with bounties of 500 points, never, I'm very unlucky, so I suppose that if you want to publish an answer based on CURSORINFO then I will accept your answer and I'll forget this question. Thanks. — ElektroStudios, Jul 03 '16 at 21:39
I understand what you mean; I prefer pure solutions too for the method/API I'm using. If you want to wait it out that's fine by me. It's just 15-25 rep we're talking about (which is about 0,65% of my total). :) — Visual Vincent, Jul 03 '16 at 22:39
Perhaps you'd get more answers if you asked on http://gamedev.stackexchange.com/ ? Or not, I don't know how popular the site is. Sadly DirectX doesn't seem to be very popular within the .NET area, only within C++/Win32. If it had better support in .NET then maybe it'd be more popular? -- I like to use DirectX too (though I'm still rather new with it). Due to its lack of popularity in the .NET area I started learning C++ instead, and created a DLL with exported methods for drawing my game. Even though I have only successfully drawn a green background yet, I still have learned **a lot**. — Visual Vincent, Jul 03 '16 at 22:52
Don't know if it captures the mouse, but [this C++ answer](http://stackoverflow.com/a/5143778/3740093) apparently captures one image. Perhaps you could mimic that and see the result? — Visual Vincent, Jul 03 '16 at 23:14
It almost seems like the mouse is drawn directly on the screen by the graphics card and that it is not bufferred or anything... Because from what I've experienced most screen recorders seem to draw the cursor themselves as many of them have _the option to include it or not_. I've used FRAPS (amongst others), and [it includes the option to hide the cursor while recording](http://ccm.net/faq/20082-fraps-hide-mouse-cursor-in-records), so it might be that it draws the cursor itself to every frame (though this is just a theory). — Visual Vincent, Jul 03 '16 at 23:15
Sorry but that code-sample is too advanced for me, I'm a DirectX novice and I'm not really aware of which DirectX interfaces are wrapped by each of the SharpDX classes (I can discover it, but just they are a lot to remember each one), anyways in the comments of that answer people says the code will not work in "X" circunstances. But don't worry anymore, the Win32 methodology that you provided is fine, it works, then I can live without a "pure" SharpDX solution. — ElektroStudios, Jul 03 '16 at 23:28
Well sadly it does not seem to be common for DirectX nor GDI to actually capture the mouse with the screenshot. So shall I write an answer including the method of P/Invoking `GetCursorInfo()` then? -- It's late here though, so you'll have to wait for it 'til the morning. :) — Visual Vincent, Jul 03 '16 at 23:39

Visual Vincent · Accepted Answer · 2016-07-04T15:43:57.350

You could utilize the WinAPI's GetCursorInfo() function to get a handle to the current cursor. By then calling Icon.FromHandle() and passing the cursor handle as a parameter you'll get the cursor as a drawable icon.

As different cursors may have different hotspots you'll have to take that into account too. For this you can use the WinAPI's GetIconInfo() function to get the X and Y-coordinates for the hotspot, then you'll just have to substract the cursor's position with the hotspot's position.

The method for drawing the cursor:

Public Shared Sub DrawCursor(ByVal Graphics As Graphics)
    Dim CursorInfo As New NativeMethods.CURSORINFO With {.cbSize = Marshal.SizeOf(GetType(NativeMethods.CURSORINFO))}

    'Get the cursor info.
    If NativeMethods.GetCursorInfo(CursorInfo) = True Then
        Using CursorIcon As System.Drawing.Icon = System.Drawing.Icon.FromHandle(CursorInfo.hCursor) 'Get the cursor icon.
            Dim IconInfo As New NativeMethods.ICONINFO

            Try
                Dim ModifierPoint As New Point(0, 0) 'Declare the modifier point (the cursor's hotspot).

                'Get the info for the cursor icon.
                If NativeMethods.GetIconInfo(CursorInfo.hCursor, IconInfo) = True Then
                    If IconInfo.fIcon = False Then 'If the cursor is an icon the hotspot will always be (0, 0).
                        ModifierPoint = New Point(IconInfo.xHotspot, IconInfo.yHotspot) 'Set the hotspot modifier.
                    End If

                End If


                'Normalize the coordinates according to the hotspot.
                Dim FinalPoint As New Point(CursorInfo.ptScreenPos.x - ModifierPoint.X, CursorInfo.ptScreenPos.y - ModifierPoint.Y)

                'Draw the cursor.
                Graphics.DrawIcon(CursorIcon, New Rectangle(FinalPoint, CursorIcon.Size))

            Finally
                'Some cleaning up...
                If IconInfo.hbmMask <> IntPtr.Zero Then
                    NativeMethods.DeleteObject(IconInfo.hbmMask)
                    IconInfo.hbmMask = IntPtr.Zero
                End If

                If IconInfo.hbmColor <> IntPtr.Zero Then
                    NativeMethods.DeleteObject(IconInfo.hbmColor)
                    IconInfo.hbmColor = IntPtr.Zero
                End If

            End Try
        End Using
    End If
End Sub

NativeMethods.vb:

Imports System.Runtime.InteropServices

Public NotInheritable Class NativeMethods
    <StructLayout(LayoutKind.Sequential)> _
    Public Structure POINT
        Public x As Integer
        Public y As Integer
    End Structure

    <StructLayout(LayoutKind.Sequential)> _
    Public Structure CURSORINFO
        Public cbSize As Integer
        Public flags As Integer
        Public hCursor As IntPtr
        Public ptScreenPos As POINT
    End Structure

    <DllImport("user32.dll")> _
    Public Shared Function GetCursorInfo(ByRef pci As CURSORINFO) As Boolean
    End Function

    <StructLayout(LayoutKind.Sequential)> _
    Public Structure ICONINFO
        Public fIcon As Boolean
        Public xHotspot As Integer
        Public yHotspot As Integer
        Public hbmMask As IntPtr
        Public hbmColor As IntPtr
    End Structure

    <DllImport("user32.dll")> _
    Public Shared Function GetIconInfo(ByVal hIcon As IntPtr, ByRef piconinfo As ICONINFO) As Boolean
    End Function

    <DllImport("gdi32.dll")> _
    Public Shared Function DeleteObject(ByVal hObject As IntPtr) As <MarshalAs(UnmanagedType.Bool)> Boolean
    End Function
End Class

If you take the resulting screenshot you'd draw the cursor onto it like this:

Using g As Graphics = Graphics.FromImage(yourImage)
    DrawCursor(g)
End Using

Include mouse cursor in screen capture using SharpDX

1 Answers1

Linked