Thanks to jewelsea's advice, I decided to look into using com.sun.glass.ui.Window to set a basic JavaFX Window's level to the maximum that Mac OSX can allow. The code snippet below shows how easy it is to use com.sun.glass.ui.Window to override the window level of a given window handle/object which can be found using com.glass.ui.Window.getWindows(). I found out how to use glass.ui.Window thanks to this answer found in another StackOverflow page.
However, there is one slight drawback (but is not an issue for me personally): decorated windows cannot be placed above the menubar even after setting the maximum window level, however, they can still be placed above the dock. If you wish to set a window above the menubar, you must first set it to Undecorated, show it, grab the window using com.sun.glass.ui.getWindows().get(window_index) and then set the window level to 3 (Max). You can set the X and Y coordinates for your window before or even after this, it seems to work both ways.
OSX Solution Code:
import javafx.application.Application;
import javafx.scene.Scene;
import javafx.scene.control.Label;
import javafx.scene.layout.Pane;
import javafx.stage.Stage;
import javafx.stage.StageStyle;
public class ontopTest extends Application{
public static void main(String[] args) {
launch(args);
}
@Override
public void start(final Stage stage){
Scene scene = new Scene(new Pane(new Label("test application")));
stage.setTitle("Override OSX Menubar test");
// Decorated window results in our window being placed below menubar even after setting window level to max.
stage.initStyle(StageStyle.UNDECORATED);
// An undecorated window fixes this and can be placed above the menubar.
stage.setWidth(300);
stage.setHeight(300);
stage.setScene(scene);
// stage.setAlwaysOnTop uneeded as com.sun.glass.ui sets window to top
stage.setY(0); // Set Y Axis to 0
// Need to show window first
stage.show();
// Print all windows running in current process
System.out.println(com.sun.glass.ui.Window.getWindows());
// Select which window to set level (window at index 0 in this case)
com.sun.glass.ui.Window.getWindows().get(0).setLevel(3);
// Set window level to 3 (Maximum)
}
}
EDIT: Although I can confirm this works on Mac OSX Sierra, I have also tested this on Arch Linux w/ KDE Plasma and this method of setting the maximum window level does not work for that platform. It simply sets the window to 'AlwaysOnTop' mode but will still be forced underneath the KDE/X11 Menubar. The only solution I know of (so far) on how to bypass the X11 menubar on Linux systems is to use the wmctrl commandline X Window manager through entering the following commands:
Add window to topmost level through selecting it by window title. This will also set the window to AlwaysOnTop
wmctrl -r "Window Title" -b add,above
Positioning the window (decorated or undecorated) to your desired (previously restricted) area, such as above the menubar, dock etc.
wmctrl -r "Window Title" -e 0,x,y,width,height
If you have a Java-based X11 Solution to setting undecorated windows above menubars/docks, please share!