6

I have an issue with matplotlib and multiprocessing. I launch a first process, where I display an image and select an area, and close the figure. Then I launch another process, where I call a graph function that is regularly updated. Up this point, eveything works fine. Then when I try to launch another process with the SAME graph function, it freeze my whole computer, BUT the background processes stil work... I only have one of these errors (it's not always the same):

error 1 :

XIO: fatal IO error 25 (Inappropriate ioctl for device) on X server ":0.0" after 4438 requests (4438 known processed) with 30 events remaining. XIO: fatal IO error 11 (Resource temporarily unavailable) on X server ":0.0" after 4443 requests (4443 known processed) with 31 events remaining. [xcb] Unknown sequence number while processing queue [xcb] Most likely this is a multi-threaded client and XInitThreads has not been called [xcb] Aborting, sorry about that. python: ../../src/xcb_io.c:274: poll_for_event: Assertion `!xcb_xlib_threads_sequence_lost' failed.

error 2 :

X Error of failed request: BadIDChoice (invalid resource ID chosen for this connection) Major opcode of failed request: 53 (X_CreatePixmap) Resource id in failed request: 0x5600299 Serial number of failed request: 4793 Current serial number in output stream: 4795 XIO: fatal IO error 25 (Inappropriate ioctl for device) on X server ":0.0" after 4788 requests (4788 known processed) with 31 events remaining. XIO: fatal IO error 25 (Inappropriate ioctl for device) on X server ":0.0" after 4793 requests (4793 known processed) with 32 events remaining.

The weird part is that I can totaly launch several process calling the graph function without any issue, it's the coupling with the first plot that make it unstable.

When trying to debug, I found out that a simple fig=plt.figure() is enough to crash everything : in fact, any call to plt ...

I read here and there that you can force matplotlib to use the agg backend and it helps with the multiprocess, but some widgets doesn't work with it so I would like to avoid this.

I don't really understand why using matplotlib in differents processes could cause problems, so if anyone could explain the reasons and/or help me with a workaround, it would be very nice.

CoMartel
  • 3,521
  • 4
  • 25
  • 48
  • Matplotlib does not work well with multiprocessing. Any chance that you can refactor your code to only work in a single process, e.g. by doing calculations in a separate process and sending the result to the main process that does the plotting? – David Zwicker Jul 11 '15 at 02:09
  • no, because the whole point is to plot data while I'm acquiring it, so it has to be a live display. But I use the `animation` function of `matplotlib` and I strongly suspect it is the cause of my issue. I will dig into it. – CoMartel Jul 15 '15 at 08:00
  • What I meant was that you do the plotting all in one process and only acquire data in the other processes. – David Zwicker Jul 15 '15 at 16:23
  • It would be nice, but in my case I use matplotlib animation to setup a camera (exposure,ZOI...), and it is this step that freeze everything. I think some of the object I use are not properly closed and conflicts other objects. – CoMartel Jul 16 '15 at 07:28

2 Answers2

1

I just had a very similar issue in which I have a class which produces plots in parallel. The first time I create a new instance of that class and run the plotting function, everything works perfectly. But if I create a new instance and plot, everything freezes.

I fixed it by writing a bash script which will in turn run a python script with the code for a single class instantiation + plot call. In other words, closing python between one plot call and the next one makes a clean slate of your working environment the computer does not freeze anymore. This is not an optimal solution, but it's working :)

Qerubin
  • 212
  • 4
  • 12
0

Since I cannot comment, a small tip as answer. If you are on a Linux machine or you have access via ssh, you can unfreeze you computer by switching to a terminal and kill all processes of the script. That will enable the input on X again. If you do not have any other python processes, the easiest way is

killall python # python3 if you use version 3

or if you want to be careful

ps as | grep python
kill # add PID numbers of processes here
hr87
  • 1,775
  • 1
  • 11
  • 17