Display Animations in Google Colab

Question

I'm trying to display animations in Google Colab. Specifically, I would like to animate a numpy array with cv2, eg drawing lines in a frame-based manner, and show the output in the cell. The closest I got was this, you can try this code in Colab:

from google.colab.patches import cv2_imshow
import IPython

from PIL import Image
import numpy as np
import cv2 as cv
import time

# Create a black image
img = np.zeros((512,512,3), np.uint8)
# Draw a diagonal blue line with thickness of 5 px
cv.line(img,(0,0),(511,511),(255,0,0),5)

cv2_imshow(img)

for i in range(100):
    cv.line(img,(i,0),(511,511),(255,0,0),5)
    cv2_imshow(img)
    IPython.display.clear_output(wait=True)
    time.sleep(1/60.0)

At some point of course this should happen without time.sleep, but with repeated callbacks so we don't block any other code execution. However, as you can see, the output flickers and is not smooth at all.

Here are a couple things I've tried:

ipycanvas. This is great in a local Jupyter notebook and is based on HTML5 canvas. It is a bit annoying to get the image data from javascript back to python, but it's possible. However, this does not run in Google Colab. https://ipycanvas.readthedocs.io/
Matplotlib animations. eg this (not mine): https://colab.research.google.com/drive/1lnl5UPFWVPrryaZZgEzd0theI6S94c3X#scrollTo=QLRBwgFqdr83 This is alright. However, it renders the whole animation before displaying it, which is not what I want. Especially, I want to be able to add some interactivity to animations, which this limitation rules out (eg clicking in the image or some button to make something happen in the animation).
Some way of explicitly creating an HTML5 canvas in javascript, eg as suggested here: IPython: Adding Javascript scripts to IPython notebook However, I'd like all my code to be python, especially my data to be numpy arrays or PIL images.

Any suggestions?

Hagrid67 · Answer 1 · 2020-09-23T14:12:08.243

Here's an example using ipywidgets.Image. This approach doesn't flicker like using clear_output, but the updates seem pretty slow. This might be to do with the fact we're running remotely from Colab - it has to send image updates over the net. Looks like I'm getting 2 or 3 per second, and it seems like it "batches up" or discards intervening updates, rather than waiting for each one.

It's pretty smooth running locally on regular Jupyter.

Hope someone can improve on this - it's something we want to do as well :)

import ipywidgets as ipw
from IPython import display
import numpy as np
import PIL
from io import BytesIO
import time

# image size
h,w = 200,300

# Make an Image Widget and display it
wIm = ipw.Image()
display.display(wIm)

# Make an RGBA array for the image
g3 = np.zeros((h,w,4), dtype=np.uint8)
g3[:,:,3] = 255  # opacity
g3[:,:,0:3] = 0  # color black

p = np.array([h//2,w//2], dtype=int)

for i in range(1000):
  # Draw a coloured spiral
  r = i/10
  theta=i/20
  p2 = p + r * np.array([ np.cos(theta), np.sin(theta) ])
  (y,x) = p2.astype(int)
  rgb = np.array([100+r, 100*(1+np.sin(theta)), 100*(1+np.cos(theta))], dtype=np.uint8)
  g3[y:y+8, x:x+2, 0:3] = rgb
  
  # convert numpy to PIL to png-format bytes
  pilIm = PIL.Image.fromarray(g3, mode="RGBA")
  with BytesIO() as fOut:
      pilIm.save(fOut, format="png")
      byPng = fOut.getvalue()
        
  # set the png bytes as the image value; 
  # this updates the image in the browser.
  wIm.value=byPng

  time.sleep(1/60)

Display Animations in Google Colab

1 Answers1