1

The graphical user interface hides mysterious mechanics under its curtain. It mixes 2D and 3D contexts on a single screen and allows for seamless composition of these two, much different worlds. But in what way and at which level are they actually interleaved?

Practice has shown that an OpenGL context can be embedded into a 2D widget library, and so the whole 2D interface can be backed with OpenGL. Also some applications may explore hardware acceleration while others don't (while being rendered on the same screen). Does the graphic card "know" about 2D and 3D areas on the screen and the window manager creates the illusion of a cohesive front-end? ...one can notice accelerated windows (3D, video) "hopping" to fit into 2D interface when, e.g. scrolling a web page or moving an video player across the screen.

The question seems to be trivial, but I haven't met anybody able to give me a comprehensive answer. An answer, which could enable me to embed an OpenGL context into a GTK+ application and understand why and how it is working. I've tried GtkGlExt and GLUT, but I would like to deeply understand the topic and write my own solution as a part of an academic project. I'd like to know what are the relations between X, GLX, GTK, OpenGL and window manager and how to explore this network of libraries to consciously program it.

I don't expect that someone will write here a dissertation, but I will be grateful for any indications, suggestions or links to articles on that topic.

Krzysztof Abramowicz
  • 1,556
  • 1
  • 12
  • 30
  • possible duplicate of [How does GUI output work from application to hardware level?](http://stackoverflow.com/questions/8777531/how-does-gui-output-work-from-application-to-hardware-level) – datenwolf Aug 01 '13 at 23:22
  • You're thinkting too complicated. OpenGL is just another API you can use to draw onto a regular X11 window. GTK+ itself (if running on X11) will just use the graphics primitives offered by the X server, which are X core, XRender and GLX/OpenGL. See my answer http://stackoverflow.com/a/8777891/524368 so a thorough explanation. – datenwolf Aug 01 '13 at 23:23
  • @datenwolf Thanks for your answers -- I caught the general idea of the GTK+ library graph for drawing. But how does it work in the opposite direction, i.e. how how are the native window events propagated back to application? I know how to catch them natively with Xlib or WinAPI; and how to use GLUT for that. But is there a non-GLUT, portable solution to be used, e.g. while implementing an OpenGL-enabled widget for GTK+? – Krzysztof Abramowicz Aug 05 '13 at 10:13
  • Well, most frameworks and toolkits implement a main event loop that uses the regular function of Xlib/Xcb or Win32 or whatever windowing API is used, to retrieve events from the native system and translate them into their own internal event format; then they inject them into their own, internal event propagation system. – datenwolf Aug 05 '13 at 10:17

1 Answers1

7

You're thinking much, much much too complicated. Toolkits like GTK+ or Qt add quite a layer of abstraction over somthing, that's actually rather simple: Your system's graphics device consists of a processor and some memory it can operate on. In the simplemost case the processor is the regular system CPU and the memory is the normal system memory. Modern computers feature a special purpose graphics processor (GPU), though, which has its own, high bandwidth memory.

The memory holds framebuffers. Logically a framebuffer is a 2D array of values. The GPU can be programmed to process the values in the framebuffers in a certain way. That can be used to draw into framebuffers. The monitors, displaying a picture are connected to a special piece of circuitry which continuously feeds the data of a certain framebuffer in the memory to the screen (usually called RAMDAC or CRTC). So in the GPU's memory there's a framebuffer that's directly going to the screen. If you draw there, things will appear on the screen.

A program, like the X11 server can load drivers that "know" how to program the GPU to draw graphical primitives. X11 itself defines certain graphics primitives, and extension modules can add further ones. X11 itself allows to segregate the framebuffers on the GPU memory into logical areas called Drawables. Drawables on the on-screen framebuffer are called Windows. Since logical Windows can overlap the X server also manages Z stacking, which it uses to sort the Windows for redraw. Everytime a Client wants to draw to some Window that X11 server will tell the GPU, that drawing operations will modify only those pixels of the framebuffer, of which the Window drawn to is visible (this is called "Pixel Ownership Test"). The X11 server will also create Drawables (i.e. framebuffers) that are not part of the on-screen framebuffer memory area. Those are called PBuffers or Pixmaps in X11 terminology (also with a special extension its possible to move a Window off-screen as well).

However all those Drawables are just memory. Technically those are Canvas to draw on with something. This something is called "graphics primitives". X11 itself provides a certain set, named X core. Also there's a de-facto standard extension called XRender which provides primitives not found in X core. However neither X11 core nor XRender provide graphics primitives with which the impression of a 3D drawing could be generated. So there's another extension, called GLX which teaches the X11 server another set of graphics primitives, namely in the form of OpenGL.

However X core, XRender and GLX/OpenGL are all just different pens, brushes and pencils that all operate on the same kind of Canvas, namely a simply framebuffer manages by X11.

And what do toolkits like Qt or GTK+ then? Well, they use X11 and the graphics primitives it provides to actually draw widgets, like Buttons, Menus and stuff like that, which X11 doesn't know about.

datenwolf
  • 159,371
  • 13
  • 185
  • 298