1

Setup

I am working on an application using Qt and Python with threads and experiance occational core dumps. I stripped down my code to a minimum working example and would like to know, what might be causing the core dumps. My application has the following goals:

  • I need to handle a compuational expensive task in a separat thread. I am using the python threading module for the thread creation.
  • The logging that occurs in that thread should be visualized in the GUI
  • The thread should send data to the GUI that should also be visualized

My setup is as follows:

Logging

I created a new logging handler that is attached to the logger and sends the log message via a queue to the main thread:

class TextEditHandler(logging.Handler):
    def __init__(self, queue):
        super(TextEditHandler, self).__init__(logging.INFO)
        self.queue = queue

    def emit(self, record):
        self.queue.put(self.format(record))

GUI

I am using a QDialog for the GUI that is created from the main window of my application. The QDialog has the following tasks:

  • It holds the QTextEdit which displays the log messages and data
  • It creates the log queue and attaches the TextEditHandler to the logger and passes the log queue to the TextEditHandler
  • It creates a QTimer that triggers in a fixed intervall and checks the log queue as well as the thread queue for new log messages/data.
  • It creates the thread as well as a thread queue in which the data is sent from the thread.

In code terms it looks like this:

class Dialog(QtGui.QDialog):
    def __init__(self):
        super(Dialog, self).__init__()

        self.logWidget = QtGui.QTextEdit()
        self.logWidget.setReadOnly(True)

        layout = QtGui.QHBoxLayout()
        layout.addWidget(self.logWidget)
        self.setLayout(layout)

        self.loggingQueue = Queue.Queue()
        self.loggingHandler = TextEditHandler(self.loggingQueue)
        logging.getLogger().addHandler(self.loggingHandler)

        self.timer = QtCore.QTimer()
        self.timer.timeout.connect(self.TimerCallback)
        self.timer.start(1)

        self.threadQueue = Queue.Queue()
        self.thread = threading.Thread(target=self.ThreadCallback, args=(self.threadQueue,))
        self.thread.daemon = True
        self.thread.start()

Thread

For completeness the thread in this minimum working example looks like this:

@classmethod
def ThreadCallback(cls, queue):
    for i in range(1000):
        time.sleep(0.001)
        logging.info("Log message {}".format(i))
        queue.put(i ** 2)

Timer

The timer call back methods should do the following:

  • It should check the logging queue for new log messages and pass them to the QTextEdit widget.
  • Also the thread queue should be checked and received data handled

In order for the timer to not block I need to check the queues non blocking (queue.get(False)). In code terms it looks like this:

def TimerCallback(self):
    try:
        log = self.loggingQueue.get(False)
        self.logWidget.append(log)
        cursor = self.logWidget.textCursor()
        cursor.movePosition(QtGui.QTextCursor.End)
        self.logWidget.setTextCursor(cursor)
    except Queue.Empty:
        pass

    try:
        data = self.threadQueue.get(False)
        print("Data: {}".format(data))
    except Queue.Empty:
        return

Missing Code for MWE

import logging
from PySide import QtCore, QtGui
import Queue
import threading
import time

if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO)
    app = QtGui.QApplication(["test"])
    dialog = Dialog()
    dialog.show()
    app.exec_()

Questions

Round about every 10-15 times a core dump occures and I have a feeling it has something to do with my threading implementation. I am fairly new to multi threaded programming and would appreciate any hint, help or suggestion you might have. I also gave the QThread class a try but found the python threads easier to comprehend.

Miscellaneous

  • Python version: 2.7.14
  • PySide version: 1.2.4
  • The code is run in a normal console, if it is run from inside the IDE (PyCharm) the problem is not reproducable.

Edit

I switched to PyQt4 as three_pineapples suggested and the core dumps appear to be fixed.

Woltan
  • 13,723
  • 15
  • 78
  • 104
  • 1
    Please read the guidance on how to provide a [mcve]. Descriptions of code and its goals are of no use. We need a complete runnable script that reproduces the problem. – ekhumoro Feb 28 '18 at 18:59
  • I've cobbled together an example script based on the snippets provided in the question (seemed to be a missing line setting default log level and of course Qt application instantiation) and I can't reproduce any crashing with PyQt 4.11.1 and Python 2.7.12. – three_pineapples Mar 01 '18 at 04:38
  • @ekhumoro I added the missing lines of code for my MWE. Thank your for the link, I didn't know of an existing FAQ about MWE. – Woltan Mar 01 '18 at 06:32
  • @three_pineapples The problem is, that the core dump does not occure on every execution of the code but only every 10-20th time. – Woltan Mar 01 '18 at 06:33
  • @Woltan I did see that you wrote that. I tested it dozens of times and it didn't crash once. I should also mention this was on windows. What platform and library versions are you using? – three_pineapples Mar 01 '18 at 07:15
  • @three_pineapples I am running the code above on Linux. Thank you for testing it! Do you think conceptually the method is sound? Using a queue to pump log messages and data to the main thread displaying it in Qt using a timer? – Woltan Mar 01 '18 at 09:20
  • @Woltan I couldn't see anything immediately wrong with it. It's not too disimilar from my approach [here](https://stackoverflow.com/a/21071865/1994235) for stderr/stdout except I exchange your timer for a Qthread that blocks on reading the queue and posts a signal to the Qt event loop. I just noticed you tagged this with "PySide" and I suspect the problem lies there. Please test with PyQt instead (API is basically the same) to see if I'm right. PySide is notorious for bugs like this. – three_pineapples Mar 01 '18 at 10:16
  • @Woltan. What *specific* versions of python, pyside and qt are you using, and on what platform? How *exactly* are you running this script? Does "10-15 times" mean within a single program run, or separate program runs? Are you running the code in an ide or debugger, or just a normal console? Did you thoroughly test *the exact code in your question* to make sure it reliably reproduces the problem? – ekhumoro Mar 01 '18 at 20:02
  • @ekhumoro I added a misc section to my question to more clearly reply to your questions. 10-15 times means on separate program runs. The minimum working example above does reproduce the core dump but on a much rarer case than the actual program. – Woltan Mar 06 '18 at 07:34
  • 1
    @Woltan. It looks like there is some kind of internal garbage-collection issue that is specific to pyside. If you want to find out more, you will have to build a debug version of pyside (and maybe qt as well), so you can get a proper stack-trace using a tool like gdb. But it will probably be a wild-goose chase. The real problem is that you are relying on libraries that are no longer officially supported or maintained. Switching to pyqt4 would be better, but even that ultimately relies on qt4 (which is obsolete legacy code). And pretty soon (2020), the same will be true of python-2.7. – ekhumoro Mar 06 '18 at 18:38
  • @ekhumoro Thank you for your evaluation. I switched to PyQt4 and the core dumps almost dissapeared. They are much rarer now. – Woltan Mar 09 '18 at 05:16

0 Answers0