5

My program is taking screenshots of other application windows to automate some tasks on them. Those windows can be hidden offscreen or obscured by other windows from time to time.

To reduce clutter, I have removed any error checking from the code listings. I am preparing the both types of screenshots with

// Get size of the target window.
RECT clientRect;
GetClientRect(hwnd, &clientRect);
int width = clientRect.right - clientRect.left;
int height = clientRect.bottom - clientRect.top;
// Create a memory device context.
HDC windowDC = GetDC(hwnd);
HDC memoryDC = CreateCompatibleDC(windowDC);
// Create a bitmap for rendering.
BITMAPINFO bitmapInfo;
ZeroMemory(&bitmapInfo, sizeof(BITMAPINFO));
bitmapInfo.bmiHeader.biSize = sizeof(BITMAPINFOHEADER);
bitmapInfo.bmiHeader.biWidth = width;
bitmapInfo.bmiHeader.biHeight = -height;
bitmapInfo.bmiHeader.biPlanes = 1;
bitmapInfo.bmiHeader.biBitCount = 32;
bitmapInfo.bmiHeader.biCompression = BI_RGB;
bitmapInfo.bmiHeader.biSizeImage = width * height * 4;
RGBQUAD* buffer;
HBITMAP bitmap = CreateDIBSection(windowDC, &bitmapInfo, DIB_RGB_COLORS, (void**)&buffer, 0, 0);
HGDIOBJ previousObject = SelectOBject(memoryDC, bitmap);

then I am taking the actual screenshots with either

PrintWindow(hwnd, memoryDC, PW_CLIENTONLY); 
GdiFlush();

or

UpdateWindow(hwnd);
BitBlt(memoryDC, 0, 0, width, height, windowDC, 0, 0, SRCCOPY);
GdiFlush();

then I copy the buffer contents to a vector

std::vector<RGBQUAD> pixels;
pixels.resize(width * height, { 0, 0, 0, 0 });
memcpy(&pixels[0], buffer, bitmapInfo.bmiHeader.biSizeImage);

and finally I am cleaning everything up with

ReleaseDC(hwnd, windowDC);
SelectObject(memoryDC, previousObject);
DeleteDC(memoryDC);
DeleteObject(bitmap);

I have a three of questions about the code above:

  1. Do I need to call GdiFlush()? I read the documentation and did some Google research, but I am still not sure if it makes sense to call it or not.
  2. Does the call to UpdateWindow() right before the BitBlt() make a difference? Does this help, so that the Device Context contents are "more up to date"?
  3. Am I right in assuming that for PrintWindow() all the work is done from within the target application (increasing the target application's CPU usage), while BitBlt() is fully executed from within the calling thread (thus increasing the CPU usage of my own application)?
  4. Under which circumstances might any of the above functions fail? I know that BitBlt() only works for hidden windows if Desktop Composition (DWM) is enabled, but on a very small set of systems (Windows 8/10) BitBlt() and PrintWindow() seem to fail for windows for which they work just fine on other systems (even though DWM is enabled). I could not spot any patterns as to why, though.

Any information is appreciated, thanks.

Johannes Stricker
  • 1,701
  • 13
  • 23
  • 1
    I think it is perfectly fine for this to fail because hidden and / or occluded windows are not required to keep their content. – user7860670 Dec 10 '18 at 17:03
  • What are you ultimately trying to accomplish? Taking screenshots to automate some takes sounds like a *really* convoluted way to mimic UI Automation. – IInspectable Dec 10 '18 at 17:40
  • 1
    GdiFlush I think is a holdover from using GDI to write to printers and plotters. I don't know that BitBlt would be expected to succeed even with DWM enabled. PrintWindow sends a WM_PRINT message so the target application must not be crashed and must actually handle it. I would expect both methods to fail if you are trying to talk to higher integrity level? windows. – Chris Becke Dec 11 '18 at 05:15
  • Thank you for your comments so far. The thing that is confusing me is that this works reliably on most systems. It may work on one system for an application and fail on another system for the same application (and application version). Could this be related to one system having two graphic cards installed? I would also appreciate any answers to (2) and (3) - thanks. I also found this comment on github: https://github.com/mozilla/positron/blob/4b0595d06faa1617bd989a8c47231576f30ea6b2/media/webrtc/trunk/webrtc/modules/desktop_capture/window_capturer_win.cc#L243 – Johannes Stricker Dec 11 '18 at 07:29
  • Alright, I was able to find out something more. The application in question is a QT application that's running on OpenGL. The OpenGL implementation that is used ('desktop', 'angle', 'software') is chosen at runtime by QT. If I force the application to run in software mode, it works on all systems. With any of the other configurations it works on one system and fails on the other (screenshots are just black). Both systems use nVidia cards with the latest drivers. Now can anyone shed some light on what might cause that particular system to behave differently? – Johannes Stricker Dec 11 '18 at 12:57
  • `BitBlt` is a GDI call. OpenGL doesn't use GDI. Neither does DirectX. – IInspectable Dec 11 '18 at 13:45
  • I know that. Nonetheless, the functions do work on some systems with OpenGL. So my question is: why? – Johannes Stricker Dec 11 '18 at 14:03
  • 1
    Check on those systems whether they are using a software rasterizer. Regardless, what are you *really* trying to accomplish here? The question strongly hints towards doing things the wrong way. Besides all that, applications can [explicitly request, that taking a screenshot won't work](https://learn.microsoft.com/en-us/windows/desktop/api/winuser/nf-winuser-setwindowdisplayaffinity). – IInspectable Dec 12 '18 at 12:59
  • Thanks a lot for your comment. Is there any way to find out which OpenGL implementation is being used, or if a software rasterizer is being used? I tried to inspect the process with Process Explorer, which shows me that opengl32.dll from my SysWOW64 folder is being loaded, but how do I know which implementation that uses? I really don't think there is a better way to do what I am doing, which is rather complicated to explain, though. – Johannes Stricker Dec 13 '18 at 13:58
  • If it's complicated to explain, it may well be that you haven't understood the problem you are trying to solve. At any rate, the solution you have opted for is *guaranteed* to fail at some point, one way or another. You can either try to explain the problem, or give up and accept, that your solution isn't. – IInspectable Dec 13 '18 at 16:18

2 Answers2

5

finally, after some hours of investigation, I found a solution for that issue: It's sufficient to call the following command within the ACTIVATE event of the form to be imaged (example in VB coding):

Call SetWindowLong(me.hwnd, GWL_EXSTYLE, WS_EX_LAYERED)

Whereas this command is defined as follows:

Private Declare Function SetWindowLong Lib "User32" Alias "SetWindowLongA" (ByVal hwnd As Long, ByVal nIndex As Long, ByVal dwNewLong As Long) As Long

Private Const GWL_EXSTYLE = (-20)

Private Const WS_EX_LAYERED = &H80000

Please try this!

Best regards, Bambi66

amanb
  • 5,276
  • 3
  • 19
  • 38
Bambi66
  • 74
  • 1
  • 2
  • I have just tested this with notepad and it really seems to work. I'll run some more tests next week! I didn't really believe that anyone would come up with a solution to this, so I am really grateful and hope that this turns out to be working not only for notepad. Thanks! – Johannes Stricker Apr 20 '19 at 16:39
  • Thank you it works great ! (Even if I don't know why) – KiwiJaune Apr 23 '19 at 09:39
  • 2
    Unfortunately, this doesn't work for all windows. The window simply does not get updated anymore after `WS_EX_LAYERED` has been set. At first, I thought it's because the window uses `CS_CLASSDC` or `CS_OWNDC`, but these styles are not set. I've tried calling `SetLayeredWindowAttributes()` without success. Any ideas? – Johannes Stricker Apr 23 '19 at 11:46
  • Sorry, I cannot edit my comment, but apparently, the window I am talking about is from an Adobe Air application. – Johannes Stricker Apr 23 '19 at 12:05
  • I noticed the same thing on main form of an application, but not on other forms. I don't know what conclude with that. (And in VB.NET, nor in C#.........) – KiwiJaune Apr 23 '19 at 15:17
  • @bambi66 Amazing!!! You saved my world. Not sure how long it took you to find this solution. Thank you so much!!!!! – tmighty Feb 22 '21 at 03:07
  • does not work on windows 10 1809 and newer – Maria Aug 16 '21 at 16:10
4

After more than two months I finally realized that this only occurs with the Windows update 1809 from late 2018. Apparently, Windows changed the way it handles clipping with that update so that the Device Context contents are no longer updated for parts of windows that are located offscreen.

To answer the individual questions:

1) GdiFlush doesn't seem to make a difference, at least from what I can tell so far.

2) Still not 100% sure about it, but I also think it doesn't make a difference.

3) Still no idea.

4) See answer above.

Johannes Stricker
  • 1,701
  • 13
  • 23
  • 2) It makes a difference for 'newly created' windows https://stackoverflow.com/a/73510627/10280599 – Osama Aug 27 '22 at 11:47