Given a hwnd, determine if that window is not hidden by other windows (Z-Ordering)

Question

I have a NativeWindow, I've overridden the WndProc function to process WM_WINDOWPOSCHANGING message for when moving my window I'll adhere it to the borders of the nearest window in the desktop.

The problem I have is that my window is docking to windows that are in the background of other top-windows, for example if I have an explorer window open and other window below the explorer window, then my window can dock to the window that is below of the other window, which is a z-order level lower than the explorer window. I want to avoid this.

A demostration of the problem:

In the GIF above, there is my window (Form1), the Visual Studio IDE window, an explorer window, and a window of an application with name "Hot Corners". When I send the "Hot Corners" window to background, I still can adhere my window to "Hot corners" borders. I want to avoid this.

Note the debug info in the captured Output window of Visual Studio.

---

I read about Z-Ordering on Wikipedia, and I also seen this example and the MSDN docs here and here, however, I still not understand how to achieve this.

When I attempt to adhere my window to other window, I need to determine if that target window is below other windows or isnt, to avoid the issue that I explained.

I hope that I explained good the problem and is clear what I need, in the GIF above my window shouldn't adhere to the "Hot Corners" window because is not visible, because the explorer window is above.

This is the relevant code, the method takes as arguments my window (a Form), the handle to a WINDOWPOS struct that I get when filtering the WM_WINDOWPOSCHANGING message in the WndProc procedure of my window, and the last parameter, threshold, is the minimum space required between the bounds of my window to other windows to adhere it.

Protected Overridable Sub DockToNearestWindowBorder(ByVal window As IWin32Window,
                                                    ByVal windowPosHandle As IntPtr,
                                                    ByVal threshold As Integer)

    Dim windowPos As WindowPos =
        CType(Marshal.PtrToStructure(windowPosHandle, GetType(WindowPos)), WindowPos)

    If (windowPos.Y = 0) OrElse (windowPos.X = 0) Then
        ' Nothing to do.
        Exit Sub
    End If

    ' Enumerate all the visible windows in the current desktop.
    Dim desktopWindows As New List(Of IntPtr)()

    Dim callBack As EnumWindowsProc =
        Function(hwnd As IntPtr, lParam As IntPtr) As Boolean
            If (NativeMethods.IsWindowVisible(hwnd)) Then
                desktopWindows.Add(hwnd)
            End If
            Return True
        End Function

    If (NativeMethods.EnumDesktopWindows(IntPtr.Zero, callBack, IntPtr.Zero)) Then

        ' Window rectangles
        Dim srcRect As Rectangle
        Dim tgtRect As Rectangle

        NativeMethods.GetWindowRect(window.Handle, srcRect)

        For Each hwnd As IntPtr In desktopWindows

            ' This is just for testing purposes.
            Dim pid As Integer
            NativeMethods.GetWindowThreadProcessId(hwnd, pid)
            If Process.GetProcessById(pid).ProcessName.EndsWith("vshost") Then
                Continue For
            End If

            NativeMethods.GetWindowRect(hwnd, tgtRect)

            ' Right border of the source window
            If ((windowPos.X + srcRect.Width) <= (tgtRect.Left + threshold)) AndAlso
               ((windowPos.X + srcRect.Width) >= (tgtRect.Left - threshold)) AndAlso
               ((windowPos.Y) <= (tgtRect.Y + tgtRect.Height)) AndAlso
               ((windowPos.Y + srcRect.Height) >= (tgtRect.Y)) Then

                    windowPos.X = (tgtRect.Left - srcRect.Width)
                    Console.WriteLine("Window adhered to: " & Process.GetProcessById(pid).ProcessName)

               ' This is not working as expected.
               ' If hwnd = NativeMethods.GetWindow(window.Handle, GetWindowCmd.HwndNext) Then
               '     windowPos.X = (tgtRect.Left - srcRect.Width)
               '     Exit For
               ' End If

            End If

        Next hwnd

    End If

    ' Marshal it back.
    Marshal.StructureToPtr(structure:=windowPos, ptr:=windowPosHandle, fDeleteOld:=True)

End Sub

Note that in the code above I only shown the threatment to adhere the right border of my window to other windows, this is to avoid increasing the code for this question, and the same reason for the missing P/Invokes.

Answer 1

Given a window handle, you should be able to determine whether the window is fully or partially obscured by other windows using a few Win32 functions:

1. Call GetWindowDC() to retrieve a device context handle (HDC) that includes the entire window, including the non-client areas (e.g., title bar, menus, borders, etc.)

2. Call GetClipBox() with the HDC returned above. This will return the smallest bounding rectangle that is actually visible (i.e., on screen and not covered by other windows). Also, the return value can tell you whether the window is completely obscured (NULLREGION).

3. Don't forget to call ReleaseDC().

API reference: https://msdn.microsoft.com/en-us/library/dd144865%28v=vs.85%29.aspx

Answer 2

Use GetWindowPlacement to avoid checking a minimized window:

Public Class YourUtilityClass

    Private Declare Function GetWindowPlacement Lib "user32" (ByVal hwnd As IntPtr, ByRef lpwndpl As WINDOWPLACEMENT) As Integer

    Private Structure WINDOWPLACEMENT
        Public Length As Integer
        Public flags As Integer
        Public showCmd As Integer
        Public ptMinPosition As POINTAPI
        Public ptMaxPosition As POINTAPI
        Public rcNormalPosition As RECT
    End Structure

    Private Structure POINTAPI
        Public x As Integer
        Public y As Integer
    End Structure

    Private Structure RECT
        Public Left As Integer
        Public Top As Integer
        Public Right As Integer
        Public Bottom As Integer
    End Structure

    Enum Placements
        Normal = 1
        Minimized = 2
        Maximized = 3
    End Enum


    Shared Function GetPlacement(ByVal hwnd As IntPtr) As Placements

        Dim wpTemp As WINDOWPLACEMENT

        wpTemp.Length = System.Runtime.InteropServices.Marshal.SizeOf(wpTemp)
        Return CType(GetWindowPlacement(hwnd, wpTemp), Placements)

    End Function
End Class

...

For Each hwnd As IntPtr In desktopWindows
   If YourUtilityClass.GetPlacement(hwnd) != YourUtilityClass.Placements.Normal Then
      Continue For
   End If

Warning: code not tested.

Api reference: https://msdn.microsoft.com/en-us/library/windows/desktop/ms633518(v=vs.85).aspx

Answer 3

The first thing you need to do (and the most difficult) is to find all the "real" non minimized (use IsIconic function) visible windows. What I mean is, if you use EnumWindows or EnumDesctopWindows you get a banch of unwanted windows such as: the windows start icon, taskbar, program manager etc which are all visible and not minimized. If you override this problem everything else is very easy: Store the handles as you did in an array. The order is from top to bottom in z-order eg.

arrayHandles[0]->top, arrayHandles[1]->one level bellow etc...

Then take the rectangles and store them in an another array. The final step is to exclude the windows that are abstracted (apparently rect0 is Ok):

check rect1 and rect0 if intersect. If true then remove rect1
check rect2 with rect1 and rect0
check rect3 with rect2 and rect1 and rect0
....

In the end you have an array of clean rects which you can check while you are moving your window.

Important: all the above steps you do it only once, the moment you start dragging your window, which is in the WM_ENTERSIZEMOVE message.

Code for the rectangle intersection (rect0 and rect1 in C code, I am too lazy):

HRGN rgn1 = {0, 0, 0, 0}, rgn2 = {0, 0, 0, 0}, rgn3 = {0, 0, 0, 0};

rgn1 = CreateRectRgn(rect0.left, rect0.top, rect0.right, rect0.bottom);

rgn2 = CreateRectRgn(rect1.left, rect1.top, rect1.right, rect1.bottom);

rgn3 = CreateRectRgn(0, 0, 0, 0);

int rslt = CombineRgn(rgn3, rgn1, rgn2, RGN_AND);

if( rslt == ERROR ){
    printf("Error in combineRgn function \n");

    //do something
}

if( rslt != NULLREGION ){
    //They DONT intersect
}

DeleteObject(rgn1);
DeleteObject(rgn2);
DeleteObject(rgn3);