26

When I try to run over 128 current OpenGLX rendering contexts on individual threads, the call to glXMakeCurrent starts failing.

Display *display = XOpenDisplay(":0")
Window root_win = RootWindow(display, screen);
Window win = XCreateWindow (display, root_win, ...)
GLXContext context = glXCreateContext(display, visinfo, 0, True);

glXMakeCurrent(display, win, context); <---- Fails here on 128th

This issue only occurs with proprietary Nvidia drivers and Nvidia GPUs. I was not able to reproduce with Intel GPUs.

Reproduction code glx.cpp:

#include <GL/gl.h>
#include <GL/glu.h>
#include <GL/glx.h>
#include <GL/glxext.h>
#include <string.h>
#include <unistd.h>
#include <thread>
#include <vector>
#include <mutex>
#include <condition_variable>
#include <chrono>

#define MAX_CONTEXTS 200;

std::mutex mutex;
std::condition_variable cond;
bool will_stop = false;

int numSuccessfulContexts = 0;
#define EXIT_IF(condition, ...) if (condition) { printf(__VA_ARGS__); exit(EXIT_FAILURE);}
#define RETURN_IF(condition, ...) if (condition) { printf(__VA_ARGS__); stop(); return; }

void stop() {
    std::lock_guard<std::mutex> lk(mutex);
    will_stop = true;
    cond.notify_all();
}

void createWindow() {
    /* Init X and GLX */
    Display *display = XOpenDisplay(":0.0");
    RETURN_IF(!display, "Cannot open X display\n");
    int screen = DefaultScreen(display);
    Window root_win = RootWindow(display, screen);
    RETURN_IF(!glXQueryExtension(display, 0, 0),"X Server doesn't support GLX extension\n");
    /* Pick an FBconfig and visual */
    static const int attributeList[] = { None };
    int fbcount;
    GLXFBConfig *fbconfig = glXChooseFBConfig(display, screen, attributeList, &fbcount);

    EXIT_IF(!fbconfig, "Failed to get GLXFBConfig\n");
    XVisualInfo *visinfo = glXGetVisualFromFBConfig(display, *fbconfig);
    EXIT_IF(!visinfo, "Failed to get XVisualInfo\n");
    /* Create the X window */
    XSetWindowAttributes winAttr ;
    winAttr.colormap = XCreateColormap(display, root_win, visinfo->visual, AllocNone);
    unsigned int mask = CWColormap;
    Window win = XCreateWindow (display, root_win, 256, 64, 320, 320, 0,
        visinfo->depth, InputOutput, visinfo->visual, mask, &winAttr) ;
    /* Create an OpenGL context and attach it to our X window */
    GLXContext context = glXCreateContext(display, visinfo, 0, True);
    EXIT_IF(!context, "Could not create GL context\n");
    RETURN_IF(! glXMakeCurrent(display, win, context), "glXMakeCurrent failed 1. \n");
    RETURN_IF(!glXIsDirect (display, glXGetCurrentContext()), "Indirect GLX rendering context obtained\n");
    RETURN_IF(!glXMakeCurrent(display, win, context), "glXMakeCurrent failed 2.\n");

    numSuccessfulContexts++;

    std::unique_lock<std::mutex> lk(mutex);
    cond.wait(lk, [] {return will_stop;});
}

int main(int argc, char *argv[]) {
    std::vector<std::thread> ts;
    printf("Starting, your computer might become unresponsive...\n");

    int maxContexts = MAX_CONTEXTS;
    while (maxContexts--) {
    ts.push_back(std::thread(&createWindow));
    }

    {
    std::unique_lock<std::mutex> lk(mutex);
    cond.wait_for(lk, std::chrono::seconds(10), []{return will_stop;});
    }

    if (!will_stop) {
    stop();
    }

    for (auto& v: ts) {
    v.join();
    }
    printf("Done. Max concurrent contexts: %d\n", numSuccessfulContexts);
    return EXIT_SUCCESS;
}

Build & Run:

g++ -std=c++11 glx.cpp -L/usr/lib/nvidia-375 -lGL -lX11 -lGLU -lGLX -lpthread -o glx && ./glx
Utku Zihnioglu
  • 4,714
  • 3
  • 38
  • 50
  • 8
    Might be a driver bug, but it's more likely that the driver simply refuses to hand out that many contexts and simply put a hard limit there. **Why are you creating that many contexts in the first place?** I see no good reason for needing that many. OpenGL contexts are kind of expensive to create and switching between them ain't cheap either. – datenwolf Mar 02 '17 at 09:52
  • 5
    Oh, and why are you creating that many threads, too? Unless you have a machine with over 100 CPU threads this makes no sense. – datenwolf Mar 02 '17 at 09:55
  • You are right. Using so many threads/contexts definitely is not the most optimal but its just how the complete rendering pipeline is currently implemented. Most of the threads will be sleeping most of the time. If we cannot find a solution to this limitation we will have to explore options on reducing thread/context counts. – Utku Zihnioglu Mar 02 '17 at 18:56
  • It feels like the limitation comes from the drivers. We tried with different Nvidia Tesla/GTX GPUs with same results. – Utku Zihnioglu Mar 03 '17 at 17:39
  • What is preventing you from sharing a small pool of OpenGL contexts between several threads? And what is preventing you from creating a small pool of worker threads between which the steps of the pipeline are divided? Search terms: "thread work queue" – datenwolf Mar 03 '17 at 18:04
  • Nothing is limiting us other than the design of the current implementation of a large pipeline. I agree that eventually we will have to switch to a producer/consumer model where we can limit the number of threads. However, currently, each thread has a long IO blocking event where they sleep most of the time and do OpenGL rendering for a small fraction of time. Ideally, until we can switch to a proper worker threads we can have a temporary solution that does not hit to the Nvidia drivers 128 OpenGL rendering context limit. – Utku Zihnioglu Mar 06 '17 at 19:25
  • An idea: delete the context while the thread is sleeping or not rendering, saving somehow its state. Create again and restore the state when a thread needs it. – Ripi2 Mar 06 '17 at 19:56
  • @Ripi2: Do you have any idea how expensive that would be. Besides, I made the experience that NVidia's drivers can be crashed by creating/deleting a lot of contexts in short succession; probably there's some kind of delayed resource deallocation implemented and one simply exhausts internal resource limits. I don't recommend doing it. – datenwolf Mar 06 '17 at 20:46
  • @datenwolf: Of course my idea is quite far from optimal, but the OP is asking for a temporay solution until he get rid of that insane number of contexts. – Ripi2 Mar 06 '17 at 21:01
  • @Ripi2 I actually tried to release the context by doing ```glXMakeCurrent(display, NONE, NULL)``` then delete the context, unmap the surface and even delete the surface. None of the steps allowed the thread to be freed up from the 128 limit. Only thing that freed up the thread was actually closing the display by running ```XCloseDisplay(display)``` – Utku Zihnioglu Mar 06 '17 at 23:01
  • Not being able to delete a context until `XCloseDisplay(...)`makes no sense to me. Try installing an error handler and calling `XSync(...)` to see what's going on. And a shot in the dark: try using glx 1.4 `glXCreateNewContext(...)` or better `glXCreateContextAttribsARB(...)` if available – Ripi2 Mar 07 '17 at 00:10
  • 3
    Found this: http://stackoverflow.com/questions/4951370/is-there-a-limit-to-how-many-opengl-rendering-contexts-you-can-create-simultaneo – Ripi2 Mar 07 '17 at 00:22
  • Both ```glXCreateNewContext``` and ```glXCreateContextAttribsARB``` seems to be affected by the same limit. – Utku Zihnioglu Mar 07 '17 at 00:52
  • Possible duplicate of [Is there a limit to how many OpenGL rendering contexts you can create simultaneously?](http://stackoverflow.com/questions/4951370/is-there-a-limit-to-how-many-opengl-rendering-contexts-you-can-create-simultaneo) – Gert Wollny Apr 07 '17 at 20:10
  • Initially I too used multiple contexts with multiple threads only to get several issues, especially if you want that your software runs on many drivers and hardware. Actually I use only one context in the main UI thread and it's not an issue because all the heavy work is in separate threads using PBO's to upload the data to the GPU (no need to have a context in the worker thread that uses only the mapped pointer). Much better control. The trick is that none of the gl calls in the UI thread are blocking the driver / GPU, are not sending any huge data to the GPU, and the commands are grouped. – andreaplanet Feb 01 '19 at 13:36

1 Answers1

1

As discussed in the comments, it seems you're hitting a driver limitation because you're doing something highly unusual and unexpected. I'm answering this to remove it from the list of unanswered questions.

Jonathan Olson
  • 1,166
  • 9
  • 19
  • I agree with you. That said, this question seemed to have gathered enough feedback/demand. I would rather keep the question open in case someone comes up with a resolution or a solid work-around. – Utku Zihnioglu Aug 07 '18 at 21:56