Embedded python crashes upon import of matplotlib.pyplot

Question

I am messing about, trying to build something similar to IPython/Jupyter notebooks. I'm writing my application in QT5, so much of this is related to 'embedding' Python in a native application.

I figured out how to embed python and how to allow it to execute scripts entered by the user. I would like to be able to use plotting libraries (such as matplotlib), and display their output in my application. (in fact, the thing I am trying to do appears to be very similar to what is described in this question).

However, when I try to import the plotting library using import matplotlib.pyplot, my application segfaults (I tried debugging, but the crash is not in my code, so I can't get anything sensible out of it).

The code I use to initialize the embedded Python, and to run arbitrary scripts is shown at the bottom of this question.

I can import other libraries (such as sys or numpy) fine. I can import matplotlib fine. But when I try to import matplotlib.pyplot, it segfaults.

Does anyone have any suggestions?

EDIT: I have determined that the cause lies (for some reason) with me using QT. When I compile a simple C or C++ program that imports matplotlib, it does not segfault...

My code:

#include "pythoninteractor.h"

#include <QString>
#include <Python.h>
#include <string>
#include <QList>

PythonInteractor::PythonInteractor()
{ 
    this->pyOutput_redir =
"import sys\n\
class CatchOutErr:\n\
    def __init__(self):\n\
        self.value = ''\n\
    def write(self, txt):\n\
        self.value += txt\n\
catchOutErr = CatchOutErr()\n\
sys.stdout = catchOutErr\n\
sys.stderr = catchOutErr\n\
"; //this is python code to redirect stdouts/stderr


    QString paths[] = {"",
                       "/home/tcpie/anaconda3/lib/python35.zip",
                       "/home/tcpie/anaconda3/lib/python3.5",
                       "/home/tcpie/anaconda3/lib/python3.5/plat-linux",
                       "/home/tcpie/anaconda3/lib/python3.5/lib-dynload",
                       "/home/tcpie/anaconda3/lib/python3.5/site-packages",};

    Py_SetProgramName(L"qt-notepad-tut");
    Py_Initialize();

    PyObject *pModule = PyImport_AddModule("__main__"); //create main module
    PyRun_SimpleString(this->pyOutput_redir.toStdString().c_str()); //invoke code to redirect

    PyObject *sys_path;
    PyObject *path;

    sys_path = PySys_GetObject("path");
    if (sys_path == NULL)
        return;

    PySequence_DelSlice(sys_path, 0, PySequence_Length(sys_path));
    for (size_t i = 0; i < sizeof(paths) / sizeof(QString); i++) {
        path = PyUnicode_FromString(paths[i].toStdString().c_str());

        if (path == NULL)
            continue;

        if (PyList_Append(sys_path, path) < 0)
            continue;
    }
}

QString PythonInteractor::run_script(QString script)
{
    QString ret = "";
    PyObject *pModule = PyImport_AddModule("__main__");
    PyRun_SimpleString(script.toStdString().c_str());
    PyErr_Print(); //make python print any errors

    PyObject *catcher = PyObject_GetAttrString(pModule,"catchOutErr"); //get our catchOutErr created above

    if (catcher == NULL) {
        Py_Finalize();
        return ret;
    }

    PyObject *output = PyObject_GetAttrString(catcher,"value"); //get the stdout and stderr from our catchOutErr object

    if (output == NULL) {
        return ret;
    }

    ret = QString(PyUnicode_AsUTF8(output));
    return ret;
}

I want to see if I can make something similar to IPython's notebooks, but in a native application. — tcpie, Jul 24 '16 at 01:27
matplotlib.pyplot might not be a good idea. If I remember correctly, it tries to create its own window. The backend should probably be `Agg`, `Cairo` or `Svg' (`matplotlib.use('Svg')) i.e. nongraphical to produce jpg, png or svg. — J.J. Hakala, Jul 24 '16 at 08:08
Ok, I get you. Thanks for the suggestions, I will try those. — tcpie, Jul 24 '16 at 17:49

score 2 · Accepted Answer · answered Jul 24 '16 at 06:58

The reason of this crash turned out to be a conflict between QT versions.

First of all, the issue can be reproduced using the following minimal code. Commenting out the "Q_OBJECT" line in main.h prevents the crash in all cases.

File main.h:

#ifndef MAIN_H
#define MAIN_H

#include <QMainWindow>

class test : public QMainWindow
{
    Q_OBJECT // Commenting out this line prevents the crash
};

#endif // MAIN_H

File main.cpp:

#include <Python.h>
#include "main.h"

int main()
{
    Py_Initialize();
    PyRun_SimpleString("import matplotlib.pyplot as plt");    
    PyRun_SimpleString("print('If we are here, we did not crash')");

    Py_Finalize();
    return 0;
}

I am running Python3 through Anaconda. However, I had installed QT5 through my package-manager (in my case: apt-get on Ubuntu). I suspect the issue lies with the matplotlib of my Anaconda install using a different QT5 version than the one I had installed through my package-manager.

The fix is easy: installing matplotlib through my package-manager fixes the issue! (on my Ubuntu system: sudo apt-get install python3-matplotlib)

Embedded python crashes upon import of matplotlib.pyplot

1 Answers1