2

I am attempting to create a transparent borderless JavaFX window/overlay on Mac OSX to allow users to select an area of the screen via drawing a rectangle and then screenshotting that area.

In order to do this on OSX, I must somehow configure my JavaFX window to float above the Mac OSX Dock/Menubar, as the window is automatically forced underneath both by default. By this, I mean I cannot drag a window over the dock or menu bar as it is forced underneath it, making a proper overlay very difficult to achieve.

How can I configure a JavaFX window's level similar to that of the NSWindow level via Cocoa?

The screenshots below (taken from a similar Electron-related post) highlight the issue and what I aim to achieve:

Current Result: https://i.stack.imgur.com/nmGvG.png

Intented Result: https://i.stack.imgur.com/kD6MR.png

denbot
  • 21
  • 5
  • Can you supply the [mcve] code used to generate the current result please? – jewelsea Jul 26 '17 at 23:07
  • I get the feeling that the only way to accomplish this will be to write some native code that [gets the OS X equivalent of the HWND handle](https://stackoverflow.com/questions/38175031/is-it-somehow-possible-to-get-a-hwnd-of-a-javafx-window-in-java-9) and manipulates the [NSWindow level as appropriate](https://developer.apple.com/documentation/appkit/nswindow/1419511-level). I don't think you are going to find anything in the public JavaFX API to directly support this feature. – jewelsea Jul 26 '17 at 23:25
  • There is some code to accomplish this in [com.sun.glass.ui.Window](http://hg.openjdk.java.net/openjfx/9/rt/file/dc2bda380efe/modules/javafx.graphics/src/main/java/com/sun/glass/ui/Window.java#l186), it has a `setLevel(Level.TOPMOST)` method, which is "A very topmost window level. May cover system UI elements such as dock, taskbar, etc.". As this class is package private in the `com.sun` hierarchy it is not public API (so is not directly supported and may not work in future Java versions) and I don't know how you would invoke the functionality from your application. – jewelsea Jul 27 '17 at 00:22
  • I notice that clicking the green expand lozenge covers the entire screen, so maybe try invoking `Application::requestToggleFullScreen`, for [example](https://stackoverflow.com/a/30308671/230513). – trashgod Jul 27 '17 at 00:30
  • jewelsea - [this is a small code snippet from my overlay](https://hastebin.com/isivefuteb.cpp) demonstrating how I construct the window itself. It works perfectly on windows but, again, is forced behind the OSX Menubar/Dock on my end: I'll have a look at com.sun.glass.ui.Window, thank you. Will it be compatible with a JavaFX Stage? trashgod - I need to avoid any form of 'fullscreen' initialization so that the overlay can be spread across all monitors. If i force fullscreen on OSX, a new workspace is opened with a black background, rendering the transparent overlay useless. – denbot Jul 27 '17 at 01:27

1 Answers1

0

Thanks to jewelsea's advice, I decided to look into using com.sun.glass.ui.Window to set a basic JavaFX Window's level to the maximum that Mac OSX can allow. The code snippet below shows how easy it is to use com.sun.glass.ui.Window to override the window level of a given window handle/object which can be found using com.glass.ui.Window.getWindows(). I found out how to use glass.ui.Window thanks to this answer found in another StackOverflow page.

However, there is one slight drawback (but is not an issue for me personally): decorated windows cannot be placed above the menubar even after setting the maximum window level, however, they can still be placed above the dock. If you wish to set a window above the menubar, you must first set it to Undecorated, show it, grab the window using com.sun.glass.ui.getWindows().get(window_index) and then set the window level to 3 (Max). You can set the X and Y coordinates for your window before or even after this, it seems to work both ways.

OSX Solution Code:

import javafx.application.Application;
import javafx.scene.Scene;
import javafx.scene.control.Label;
import javafx.scene.layout.Pane;
import javafx.stage.Stage;
import javafx.stage.StageStyle;

public class ontopTest extends Application{

    public static void main(String[] args) {
        launch(args);
    }

    @Override
    public void start(final Stage stage){
        Scene scene = new Scene(new Pane(new Label("test application")));
        stage.setTitle("Override OSX Menubar test");
        // Decorated window results in our window being placed below menubar even after setting window level to max.
        stage.initStyle(StageStyle.UNDECORATED);
        // An undecorated window fixes this and can be placed above the menubar.
        stage.setWidth(300);
        stage.setHeight(300);
        stage.setScene(scene);
        // stage.setAlwaysOnTop uneeded as com.sun.glass.ui sets window to top
        stage.setY(0); // Set Y Axis to 0
        // Need to show window first
        stage.show();
        // Print all windows running in current process
        System.out.println(com.sun.glass.ui.Window.getWindows());
        // Select which window to set level (window at index 0 in this case)
        com.sun.glass.ui.Window.getWindows().get(0).setLevel(3);
        // Set window level to 3 (Maximum)
    }
}

EDIT: Although I can confirm this works on Mac OSX Sierra, I have also tested this on Arch Linux w/ KDE Plasma and this method of setting the maximum window level does not work for that platform. It simply sets the window to 'AlwaysOnTop' mode but will still be forced underneath the KDE/X11 Menubar. The only solution I know of (so far) on how to bypass the X11 menubar on Linux systems is to use the wmctrl commandline X Window manager through entering the following commands:

Add window to topmost level through selecting it by window title. This will also set the window to AlwaysOnTop

wmctrl -r "Window Title" -b add,above

Positioning the window (decorated or undecorated) to your desired (previously restricted) area, such as above the menubar, dock etc.

wmctrl -r "Window Title" -e 0,x,y,width,height

If you have a Java-based X11 Solution to setting undecorated windows above menubars/docks, please share!

denbot
  • 21
  • 5
  • This kind of hack will always be platform dependent. Be wary of using com.sun.* packages as they may change without warning or proper documentation from release to release. see JewelSea's comment above... – Eric Jul 27 '17 at 03:58