Memory leak when embedding and updating a matplotlib graph in a PyQt GUI

Question

I am trying to embed a matplotlib graph that updates every second into a PyQt GUI main window.

In my program I call an update function every second using threading.Timer via the timer function shown below. I have a problem: my program grows bigger every second - at a rate of about 1k every 4 seconds. My initial thoughts are that the append function (that returns a new array in update_figure) does not delete the old array? Is it possible this is the cause of my problem?

def update_figure(self):
    self.yAxis = np.append(self.yAxis, (getCO22()))
    self.xAxis = np.append(self.xAxis, self.i)
    # print(self.xAxis)
    if len(self.yAxis) > 10:
        self.yAxis = np.delete(self.yAxis, 0)

    if len(self.xAxis) > 10:
        self.xAxis = np.delete(self.xAxis, 0)

    self.axes.plot(self.xAxis, self.yAxis, scaley=False)
    self.axes.grid(True)

    self.i = self.i + 1

    self.draw()

This is my timer function - this is triggered by the click of a button in my PyQt GUI and then calls itself as you can see:

def timer(self):
    getCH4()
    getCO2()
    getConnectedDevices()
    self.dc.update_figure()
    t = threading.Timer(1.0, self.timer)
    t.start()

EDIT: I cant post my entire code because it requires a lot of .dll includes. So i'll try to explain what this program does.

In my GUI I want to show the my CO₂ value over time. My get_co22 function just returns a float value and I'm 100% sure this works fine. With my timer, shown above, I want to keep append a value to a matplotlib graph - the Axes object is available to me as self.axes. I try to plot the last 10 values of the data.

EDIT 2: After some discussion in chat, I tried putting the call to update_figure() in a while loop and using just one thread to call it and was able to make this minimal example http://pastebin.com/RXya6Zah. This changed the structure of the code to call update_figure() to the following:

def task(self):
    while True:
        ui.dc.update_figure()
        time.sleep(1.0)

def timer(self):
    t = Timer(1.0, self.task())
    t.start()

but now the program crashes after 5 iterations or so.

Can you craft a simple example that we can actually run and verify? For example by removing the calls to additional functions and without the timer, s.t. it consumes memory really quick? — moooeeeep, Apr 21 '15 at 08:09
What do you intend to achieve with `np.delete` calls? [Docs](http://docs.scipy.org/doc/numpy/reference/generated/numpy.delete.html) says, that it deletes subarray. So it seems like your arrays grows faster than they are shrinks. — Gill Bates, Apr 21 '15 at 08:16
I think you need to show how your plotting works. My suspicion is that every time you call self.draw(), references to the arrays and other (maybe bigger) objects are retained. — J Richard Snape, Apr 21 '15 at 08:25
@GillBates with the np.delete I make sure my array that I plot keeps holding 10 values. I append 1 value, and then delete the first value so that I get an updated array to plot — Rowan Klein Gunnewiek, Apr 21 '15 at 08:26
More succinctly - I wonder if your GUI is the problem. Try commenting out `self.draw()` and / or `self.axes.plot(...` and see if you still have the memory leak. If not - you've got a clue where the problem lies. — J Richard Snape, Apr 21 '15 at 08:26
@JRichardSnape I tried commenting self.draw() and self.axes.plot(). The program keeps growing, but it grows slower then before! — Rowan Klein Gunnewiek, Apr 21 '15 at 08:33
You have to try to simplify. I can use the portion of your code that appends to the array and then keeps it to length 10 for ever without memory leaking - It's up to several million now - array stays at length 10 and no discernible memory use. Try commenting all calls to plotting. Take the `timer` out and just call the `update_figure` from a loop etc. If you can't post all code and can't simplify / remove the ambiguity, it's going to be really hard to help. — J Richard Snape, Apr 21 '15 at 08:45
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/75784/discussion-between-j-richard-snape-and-rowan-klein-gunnewiek). — J Richard Snape, Apr 21 '15 at 09:21
possible duplicate of [How can I release memory after creating matplotlib figures](http://stackoverflow.com/questions/7101404/how-can-i-release-memory-after-creating-matplotlib-figures) — Ruggero Turra, Apr 21 '15 at 09:25
It's definitely not a duplicate of that - see the [chat discussion linked above](http://chat.stackoverflow.com/rooms/75784/discussion-between-j-richard-snape-and-rowan-klein-gunnewiek) However, the question does need quite an edit. As I suspected and we discussed in that room - it's the threading model that is at fault here. — J Richard Snape, Apr 22 '15 at 11:34

score 2 · Accepted Answer · edited Jan 18 '21 at 12:34

The problem is definitely not with how you are appending to your numpy array, or truncating it.

The problem here is with your threading model. Integrating calculation loops with a GUI control loop is difficult.

Fundamentally, you need your GUI threading to have control of when your update code is called (spawning a new thread to handle it if necessary) - so that

your code does not block the GUI updating,
the GUI updating does not block your code executing and
you don't spawn loads of threads holding multiple copies of objects (which might be where your memory leak comes from).

In this case, as your main window is controlled by PyQt4, you want to use a QTimer (see a simple example here)

So - alter your timer code to

def task(self):
    getCH4()
    getCO2()
    getConnectedDevices()
    self.dc.update_figure()

def timer(self):
    self.t = QtCore.QTimer()
    self.t.timeout.connect(self.task)
    self.t.start(1000)

and this should work. Keeping the reference to the QTimer is essential - hence self.t = QtCore.QTimer() rather than t = QtCore.QTimer(), otherwise the QTimer object will be garbage collected.

Note:

This is a summary of a long thread in chat clarifying the issue and working through several possible solutions. In particular - the OP managed to mock up a simpler runnable example here: http://pastebin.com/RXya6Zah

and the fixed version of the full runnable example is here: http://pastebin.com/gv7Cmapr

The relevant code and explanation is above, but the links might help anyone who wants to replicate / solve the issue. Note that they require PyQt4 to be installed

score 0 · Answer 2 · edited May 23 '17 at 12:22

0

if you are creating a new figure for every time this is quite common.

matplotlib do not free the figures you create, unless you ask it, somethink like:

pylab.close()

see How can I release memory after creating matplotlib figures

edited May 23 '17 at 12:22

Community

1
1

answered Apr 21 '15 at 09:19

Ruggero Turra

16,929
16
85
141

Yes - that's where I was going with my comments to the OP, but it's not clear whether they're using matplotlib. It also seems that commenting out all the plotting did not relieve the problem. – J Richard Snape Apr 21 '15 at 09:21

Memory leak when embedding and updating a matplotlib graph in a PyQt GUI

2 Answers2

Note:

Linked