Python - Take screenshot including mouse cursor

Question

I am currently trying to take a screenshot who include mouse cursor using PIL.ImageGrab.

This is my code:

import ctypes, win32gui, win32ui
from PIL import Image, ImageGrab

size = round(ctypes.windll.shcore.GetScaleFactorForDevice(0) / 100 * 32)

cursor = get_cursor()

pixdata = cursor.load()
minsize = [size, None]

width, height = cursor.size
for y in range(height):
    for x in range(width):

        if pixdata[x, y] == (0, 0, 0, 255):
            pixdata[x, y] = (0, 0, 0, 0)

        else:
            if minsize[1] == None:
                minsize[1] = y

            if x < minsize[0]:
                minsize[0] = x

ratio = ctypes.windll.shcore.GetScaleFactorForDevice(0) / 100

img = ImageGrab.grab(bbox=None, include_layered_windows=True)

pos_win = win32gui.GetCursorPos()
pos = (round(pos_win[0]*ratio), round(pos_win[1]*ratio))


img.paste(cursor, pos, cursor)

img.save("screenshot.png")

And this is my get_cursor() function:

def get_cursor():

    hcursor = win32gui.GetCursorInfo()[1]
    hdc = win32ui.CreateDCFromHandle(win32gui.GetDC(0))
    hbmp = win32ui.CreateBitmap()
    hbmp.CreateCompatibleBitmap(hdc, 36, 36)
    hdc = hdc.CreateCompatibleDC()
    hdc.SelectObject(hbmp)
    hdc.DrawIcon((0,0), hcursor)
    
    bmpinfo = hbmp.GetInfo()
    bmpbytes = hbmp.GetBitmapBits()
    bmpstr = hbmp.GetBitmapBits(True)
    cursor = Image.frombuffer('RGB', (bmpinfo['bmWidth'], bmpinfo['bmHeight']), bmpstr, 'raw', 'BGRX', 0, 1).convert("RGBA")
    
    win32gui.DestroyIcon(hcursor)    
    win32gui.DeleteObject(hbmp.GetHandle())
    hdc.DeleteDC()


    pixdata = cursor.load()
    minsize = [32, None]

    width, height = cursor.size
    for y in range(height):
        for x in range(width):

            if pixdata[x, y] == (0, 0, 0, 255):
                pixdata[x, y] = (0, 0, 0, 0)

            else:
                if minsize[1] == None:
                    minsize[1] = y

                if x < minsize[0]:
                    minsize[0] = x


    return cursor

The problem is that some cursors are not pasted at the right position because they have pixels to the left of their position like this (do not pay attention to the quality).

How can I place the cursor image correctly (or otherwise solve the problem)?

Add a delay before the image grab so that you can position the cursor in the correct place. This is similar to the delay feature in the windows snipping tool — cup, May 30 '22 at 07:43
@cup The problem is not the actual position of the cursor but how the image returned by my `get_cursor` function is placed. On both screenshots, the real cursor is exactly at the same position (and is not moving) — crazycat256, May 30 '22 at 07:51
https://github.com/BoboTiG/python-mss/issues/55#issuecomment-580481146 there is a part of code you want — Kal-1, May 31 '22 at 09:33
@Kal-1 I tried this solution but I can't save or display this image format — crazycat256, May 31 '22 at 12:31
"The problem is not the actual position of the cursor but how the image returned by my get_cursor function is placed." Yes; cursors have a "hot spot" which indicates where the OS will position the cursor image relative to the actual mouse position. It's effectively hard-coded data that depends on the image. It might be represented in the `win32gui.GetCursorInfo()` result somewhere; try reading the documentation. In the worst case, you can just hard-code the offset yourself. — Karl Knechtel, Jun 01 '22 at 20:17

score 2 · Accepted Answer · answered Jun 02 '22 at 06:39

Mouse cursors have a "hot spot" which is the position where the image is placed relative to the cursor position This "hot spot" can be obtained like this:

import win32gui

hcursor = win32gui.GetCursorInfo()[1]
hotspot = win32gui.GetIconInfo(hcursor)[1:3]

So the full code is:

import ctypes, win32gui, win32ui
from PIL import Image, ImageGrab


def get_cursor():
    hcursor = win32gui.GetCursorInfo()[1]
    hdc = win32ui.CreateDCFromHandle(win32gui.GetDC(0))
    hbmp = win32ui.CreateBitmap()
    hbmp.CreateCompatibleBitmap(hdc, 36, 36)
    hdc = hdc.CreateCompatibleDC()
    hdc.SelectObject(hbmp)
    hdc.DrawIcon((0,0), hcursor)
    
    bmpinfo = hbmp.GetInfo()
    bmpstr = hbmp.GetBitmapBits(True)
    cursor = Image.frombuffer('RGB', (bmpinfo['bmWidth'], bmpinfo['bmHeight']), bmpstr, 'raw', 'BGRX', 0, 1).convert("RGBA")
    
    win32gui.DestroyIcon(hcursor)    
    win32gui.DeleteObject(hbmp.GetHandle())
    hdc.DeleteDC()


    pixdata = cursor.load()


    width, height = cursor.size
    for y in range(height):
        for x in range(width):

            if pixdata[x, y] == (0, 0, 0, 255):
                pixdata[x, y] = (0, 0, 0, 0)


    hotspot = win32gui.GetIconInfo(hcursor)[1:3]

    return (cursor, hotspot)

cursor, (hotspotx, hotspoty) = get_cursor()
cursor.save("cursor.png")


ratio = ctypes.windll.shcore.GetScaleFactorForDevice(0) / 100

img = ImageGrab.grab(bbox=None, include_layered_windows=True)

pos_win = win32gui.GetCursorPos()
pos = (round(pos_win[0]*ratio - hotspotx), round(pos_win[1]*ratio - hotspoty))


img.paste(cursor, pos, cursor)

img.save("screenshot.png")

Python - Take screenshot including mouse cursor

1 Answers1

Linked