Capturing the screen behind the window

Question

I want to write a Windows C++ application where the contents of the window is whatever is behind the window (as if the window is transparent). That is, I want to retrieve the bounding box of my window; capture those coordinates below, and draw them on my window. Therefore it is crucial that I can exclude the window itself during the capture.

"Why not just make the window transparent?" you ask. Because the next step for me is to make modifications to that image. I want to apply some arbitrary filters on it. For example, let's just say that I want to blur that image, so that my window looks like a frosted glass.

I tried to use the magnification API sample at https://code.msdn.microsoft.com/windowsdesktop/Magnification-API-Sample-14269fd2 which actually provides me the screen contents excluding my window. However, re-rendering the image is done in a timer, which causes a very jittery image; and I couldn't figure out how to retrieve and apply arbitrary transformations to that image.

I don't know where to start and really could use some pointers at this point. Sorry if I'm approaching this from a stupid perspective.

Edit: I am adding a mock-up of what I mean:

Edit 2: Just like in the magnification API example, view would be constantly refreshed (as frequently as possible, say every 16 ms just for argument's sake). See Visolve Deflector for an example; although it does not apply any effects on the captured region.

Again, I will be modifying the image data afterwards; therefore I cannot use the Magnification API's kernel matrix support.

Answer 1

You can start by modifying MAGCOLOREFFECT . In MagnifierSample.cpp we have:

if (ret) 
{ 
    MAGCOLOREFFECT magEffectInvert =  
    {{ // MagEffectInvert 
        { -1.0f,  0.0f,  0.0f,  0.0f,  0.0f }, 
        {  0.0f, -1.0f,  0.0f,  0.0f,  0.0f }, 
        {  0.0f,  0.0f, -1.0f,  0.0f,  0.0f }, 
        {  0.0f,  0.0f,  0.0f,  1.0f,  0.0f }, 
        {  1.0f,  1.0f,  1.0f,  0.0f,  1.0f }  
    }}; 

    ret = MagSetColorEffect(hwndMag,&magEffectInvert); 
} 

Using a Color Matrix to Transform a Single Color.

For more advanced effects, you can blit contents to memory device context.

Answer 2

You did not specify if this is a one time activity or you need a continuous stream of whats behind your window (like Magnifier/etc). And if continuous, whats the frequency of updates you may need.

Anyway in either case I see two primary use cases:

1. The contents behind your app are constant: You may not believe, but most of the time the contents behind your window will not change.
2. The contents behind your window are changing/animating: this is a trickier case.

Thus if you can let go the non-constant/animated background usecase, the solution is pretty simple in both one shot and continuous stream cases:

1. Hide your application window
2. Take a screenshot, and cache it!
3. Show your app back (crop everything apart from your application main window's bounding box), and now user can apply the filter
4. Even if user changes the filter, reapply that to to cached image.
5. Track your window's WM_MOVE/WM_SIZE and repeat above process for new dimensions.

Additionally if you need to be precise, use SetWindowsHookEx for CBT/etc.

Corner cases from top of my head:

1. Notify icon/Balloon tool tips
2. Desktop background scheduling (windows third party app)
3. Application specific message boxes etc!

Hope this helps!

Answer 3

I've achieved something akin to this using the "GetForeGroundWindow" and "PrintWindow".

It's kind of involved, but here is a picture. The Image updates with it's source, but it's slow, so there is a significant lag (i.e .2 seconds=.5seconds) Rather than a blur effect I opted for a SinWave effect. Also, using GetForeGroundWindow basically means it can only copy the contents of one window. If you want to hear more just respond and I'll put together some steps and an example repo.