how to save this Matplotlib drawing as a Numpy array?

Question

I have a function that takes an image stored as a Numpy array, draws a few rectangles on it, labels them, then displays the result.

The shape of the source Numpy array is (480, 640, 3) - it's an RGB image from a camera. This probably doesn't matter a lot, but I'm just showing you an example of the data I'm working with.

This is the function:

def draw_boxes(imdata, v_boxes, v_labels, v_scores):
    fig = pyplot.imshow(imdata)
    # get the context for drawing boxes
    ax = pyplot.gca()
    # plot each box
    for i in range(len(v_boxes)):
        box = v_boxes[i]
        # get coordinates
        y1, x1, y2, x2 = box.ymin, box.xmin, box.ymax, box.xmax
        # calculate width and height of the box
        width, height = x2 - x1, y2 - y1
        # create the shape
        rect = Rectangle((x1, y1), width, height, fill=False, color='white')
        # draw the box
        ax.add_patch(rect)
        # draw text and score in top left corner
        label = "%s (%.3f)" % (v_labels[i], v_scores[i])
        ax.text(x1, y1, label, color='white')
    pyplot.show()

I would like to take the annotated image (the image with the rectangles and labels drawn on it) and extract all that as a Numpy array. Basically, return an annotated Numpy array.

I've spent a couple hours trying various solution found on Google, but nothing works. For example, I cannot do this...

fig.canvas.draw()
X = np.array(fig.canvas.renderer.buffer_rgba())

...because fig.canvas.draw() fails with:

AttributeError: 'AxesImage' object has no attribute 'canvas'

score 1 · Accepted Answer · answered Oct 02 '19 at 07:59

1

The problem is that your fig variable is not a figure but an AxesImage as the error is stating. Thus change the first line of your code with :

fig, ax = plt.subplots()
ax = plt.imshow(imdata)

The complete function is then :

def draw_boxes(imdata, v_boxes, v_labels, v_scores):
    fig, ax = plt.subplots()
    ax = plt.imshow(imdata)
    # get the context for drawing boxes
    ax = pyplot.gca()
    # plot each box
    for i in range(len(v_boxes)):
        box = v_boxes[i]
        # get coordinates
        y1, x1, y2, x2 = box.ymin, box.xmin, box.ymax, box.xmax
        # calculate width and height of the box
        width, height = x2 - x1, y2 - y1
        # create the shape
        rect = Rectangle((x1, y1), width, height, fill=False, color='white')
        # draw the box
        ax.add_patch(rect)
        # draw text and score in top left corner
        label = "%s (%.3f)" % (v_labels[i], v_scores[i])
        ax.text(x1, y1, label, color='white')
    fig.canvas.draw()
    X = np.array(fig.canvas.renderer.buffer_rgba(), dtype=float)
    return X

answered Oct 02 '19 at 07:59

Simon Delecourt

1,519
8
13

Okay, getting closer, thanks. I do get an image, but it's weird, it's like a color negative: https://i.imgur.com/GIOZzu3.png Instead, it should be an image of me looking at the computer; the yellow dots at the bottom are supposedly my dark gray shirt. If I do pyplot.show() instead, that looks fine. – Florin Andrei Oct 02 '19 at 08:29
I'm displaying the array with cv2.imshow('camera', X) in the function. – Florin Andrei Oct 02 '19 at 08:32
I get an almost perfect image with X = np.array(fig.canvas.renderer.buffer_rgba(), dtype=np.uint8). But the colors are a bit weird - the yellows are blue. – Florin Andrei Oct 02 '19 at 08:38
Found the solution: ``` X = np.array(fig.canvas.renderer.buffer_rgba(), dtype=np.uint8) cv2.imshow('camera', cv2.cvtColor(X, cv2.COLOR_BGR2RGB)) ``` – Florin Andrei Oct 02 '19 at 08:45

how to save this Matplotlib drawing as a Numpy array?

1 Answers1