0

I'm trying to make a DirectX8 application (Game Maker) chroma keyed using layered windows (WS_EX_LAYERED extended style flags, and LWA_COLORKEY method) for transparency and correct hit testing on a borderless window. However, out of all platforms I've tested it on, Windows 7 fails but specifically only when Aero Glass is turned on. The effect works for apparently one frame, and then stops working - the chroma key then comes out to the screen and the window is a full rectangle, including mouse hit testing. Turning off Aero Glass in advanced appearance, or changing to a Basic or Classic theme fixes the issue, but that's undesirable.

__declspec(dllexport) double __cdecl __gm82core_setchromakey(double enable, double color) {  
    if (enable>=0.5) {    
        SetWindowLong(window_handle,GWL_EXSTYLE,GetWindowLong(window_handle,GWL_EXSTYLE) | WS_EX_LAYERED); 
        SetLayeredWindowAttributes(window_handle,((DWORD)color)&0x00ffffff,0xff,LWA_COLORKEY);          
    } else {
        SetWindowLong(window_handle,GWL_EXSTYLE,GetWindowLong(window_handle,GWL_EXSTYLE) & ~WS_EX_LAYERED);
    }
    return 0;
}

broken effect in Win7 with Aero Glass enabled

(The variable types are all double because the develoment platform (Game Maker) only allows for doubles to be passed to native code)

  • I've read that there are issues using colors that have the same value on multiple channels like white or yellow for chroma key, and tried using colors such as rgb (253,254,255) or (201,202,203), to similar results.
  • I've read that chroma keying apparently broke in Windows 7 RC1, and we can just assume that wasn't fixed for RTM due to the experienced behavior.
  • I've read that in Win 7 and earlier, child windows cannot be chroma keyed, and I've attempted to remove the window's parent using SetParent(), to similar results (and introducing other issues in the process), despite the window's parent being disabled and invisible when the application is launched in borderless mode.
  • The effect works fine until the window is updated, and then never again, which is an issue as the program has animated controls.
  • Was verified to work in various versions of Win 10 dated as far back as 1903: effect working as intended in Win10
  • I've used an alternate method using DWM direct compositing using a 32 bit ARGB surface format, which allows per-pixel alpha transparency in all platforms, however does not allow for mouse hit testing, which is undesirable for this application due to its free-form window.
  • Even though the effect works properly in Win10, I'd like to support Win7 as well since that's a common target for the extension package this code is from.
renex
  • 1
  • 1

0 Answers0