Plt Plot to OpenCV Image/Numpy Array

Question

I have a piece of code which I use to visualize a graph:

if (visualize == True):
    # Black removed and is used for noise instead.
    unique_labels = set(db.labels_)
    colors = [plt.cm.Spectral(each)
            for each in np.linspace(0, 1, len(unique_labels))]
    for k, col in zip(unique_labels, colors):
        if k == -1:
            # Black used for noise.
            col = [0, 0, 0, 1]

        class_member_mask = (db.labels_ == k)

        xy = scaled_points[class_member_mask & core_samples_mask]
        plt.plot(xy[:, 0], xy[:, 1], 'o', markerfacecolor=tuple(col),
                markeredgecolor='k', markersize=14)

        xy = scaled_points[class_member_mask & ~core_samples_mask]
        plt.plot(xy[:, 0], xy[:, 1], 'o', markerfacecolor=tuple(col),
                markeredgecolor='k', markersize=6)

    # display the graph
    plt.title('Estimated number of clusters: %d' % n_clusters_)
    plt.show()

    # get the image into a variable that OpenCV likes
    # uh, how?

while this works, I want to have the end result (whatever is being shown) as an OpenCV image.

Since I don't even have the variable -image-, I have no idea how to achieve this.

Did anyone do something similar?

EDIT: I am actually getting close. Now I can create an OpenCV image out of a fig, but the contents are not right. The fig is empty. I wonder where I go wrong? Why doesn't it get the plt object from above and draw the actual content?

fig = plt.figure()
canvas = FigureCanvas(fig)
canvas.draw()

# convert canvas to image
graph_image = np.fromstring(fig.canvas.tostring_rgb(), dtype=np.uint8, sep='')
graph_image  = graph_image.reshape(fig.canvas.get_width_height()[::-1] + (3,))

# it still is rgb, convert to opencv's default bgr
graph_image = cv2.cvtColor(graph_image,cv2.COLOR_RGB2BGR)

Python doesn't have a `Mat` type variable, that's in C++. It think Python treats images as arrays. Edited the question title accordingly. — Schütze, Mar 20 '19 at 13:19
As confirmed here: https://stackoverflow.com/questions/7762948/how-to-convert-an-rgb-image-to-numpy-array Images in OpenCV are of type `numpy.ndarray` — Eskapp, Mar 20 '19 at 15:33
I can actually save the image using `plt.savefig('/tmp/graph' + str(self.frame_id) + ".jpg")` but I cannot assign it to a variable, i.e to a `numpy.ndarray`. I can reload the image every time but this would be too costly — Schütze, Mar 20 '19 at 15:36
Looks like you are looking for [Agg Buffer To Array](https://matplotlib.org/gallery/misc/agg_buffer_to_array.html). — ImportanceOfBeingErnest, Mar 20 '19 at 16:18

Schütze · Answer 1 · 2019-03-21T08:08:36.557

Okay, I finally got it! One has to create the fig object at the very beginning, then use the necessary plotting functions, then convert to canvas and then to OpenCV image.

EDIT: Thanks to the suggestion of @ImportanceOfBeingErnest, now the code is even more straightforward!

Here is the full code:

if (visualize == True):
    # create a figure
    fig = plt.figure()

    # Black removed and is used for noise instead.
    unique_labels = set(db.labels_)
    colors = [plt.cm.Spectral(each)
            for each in np.linspace(0, 1, len(unique_labels))]
    for k, col in zip(unique_labels, colors):
        if k == -1:
            # Black used for noise.
            col = [0, 0, 0, 1]

        class_member_mask = (db.labels_ == k)

        xy = scaled_points[class_member_mask & core_samples_mask]
        plt.plot(xy[:, 0], xy[:, 1], 'o', markerfacecolor=tuple(col),
                markeredgecolor='k', markersize=14)

        xy = scaled_points[class_member_mask & ~core_samples_mask]
        plt.plot(xy[:, 0], xy[:, 1], 'o', markerfacecolor=tuple(col),
                markeredgecolor='k', markersize=6)

    # convert it to an OpenCV image/numpy array
    canvas = FigureCanvas(fig)
    canvas.draw()

    # convert canvas to image
    graph_image = np.array(fig.canvas.get_renderer()._renderer)

    # it still is rgb, convert to opencv's default bgr
    graph_image = cv2.cvtColor(graph_image,cv2.COLOR_RGB2BGR)

This is much more complicated than what I suggested in the comments below the quesiton. It would simply be `graph_image = np.array(fig.canvas.get_renderer()._renderer)`. — ImportanceOfBeingErnest, Mar 21 '19 at 03:23

Plt Plot to OpenCV Image/Numpy Array

1 Answers1