How to render OpenAI gym in google Colab?

Question

I'm trying to use OpenAI gym in google colab. As the Notebook is running on a remote server I can not render gym's environment.

I found some solution for Jupyter notebook, however, these solutions do not work with colab as I don't have access to the remote server.

I wonder if someone knows a workaround for this that works with google Colab?

Answer 1

Korakot's answer is not correct.

You can indeed render OpenAi Gym in colaboratory, albiet kind of slowly using none other than matplotlib.

Heres how:

Install xvfb & other dependencies (Thanks to Peter for his comment)

!apt-get install x11-utils > /dev/null 2>&1 
!pip install pyglet > /dev/null 2>&1 
!apt-get install -y xvfb python-opengl > /dev/null 2>&1

As well as pyvirtual display:

!pip install gym pyvirtualdisplay > /dev/null 2>&1

then import all your libraries, including matplotlib & ipythondisplay:

import gym
import numpy as np
import matplotlib.pyplot as plt
from IPython import display as ipythondisplay

then you want to import Display from pyvirtual display & initialise your screen size, in this example 400x300... :

from pyvirtualdisplay import Display
display = Display(visible=0, size=(400, 300))
display.start()

last but not least, using gym's "rgb_array" render functionally, render to a "Screen" variable, then plot the screen variable using Matplotlib! (rendered indirectly using Ipython display)

env = gym.make("CartPole-v0")
env.reset()
prev_screen = env.render(mode='rgb_array')
plt.imshow(prev_screen)

for i in range(50):
  action = env.action_space.sample()
  obs, reward, done, info = env.step(action)
  screen = env.render(mode='rgb_array')

  plt.imshow(screen)
  ipythondisplay.clear_output(wait=True)
  ipythondisplay.display(plt.gcf())

  if done:
    break

ipythondisplay.clear_output(wait=True)
env.close()

Link to my working Colaboratory notebook demoing cartpole:

https://colab.research.google.com/drive/16gZuQlwxmxR5ZWYLZvBeq3bTdFfb1r_6

Note: not all Gym Environments support "rgb_array" render mode, but most of the basic ones do.

Answer 2

Try this :-

!apt-get install python-opengl -y

!apt install xvfb -y

!pip install pyvirtualdisplay

!pip install piglet


from pyvirtualdisplay import Display
Display().start()

import gym
from IPython import display
import matplotlib.pyplot as plt
%matplotlib inline

env = gym.make('CartPole-v0')
env.reset()
img = plt.imshow(env.render('rgb_array')) # only call this once
for _ in range(40):
    img.set_data(env.render('rgb_array')) # just update the data
    display.display(plt.gcf())
    display.clear_output(wait=True)
    action = env.action_space.sample()
    env.step(action)

This worked for me so I guess it should also work for you.

Answer 3

I recently had to solve the same problem and have written up a blog post with my solution. For ease of reference I am re-posting the TLDR; version here.

Paste this code into a cell in Colab and run it to install all of the dependencies.

%%bash

# install required system dependencies
apt-get install -y xvfb x11-utils

# install required python dependencies (might need to install additional gym extras depending)
pip install gym[box2d]==0.17.* pyvirtualdisplay==0.2.* PyOpenGL==3.1.* PyOpenGL-accelerate==3.1.*

And then start a virtual display in the background.

import pyvirtualdisplay


_display = pyvirtualdisplay.Display(visible=False,  # use False with Xvfb
                                    size=(1400, 900))
_ = _display.start()

In the blog post I also provide a sample simulation demo that demonstrates that the above actually works.

Answer 4

By far the best solution I found after spending countless hours on this issue is to record & play video. It comes UX wise very close to the real render function.

Here is a Google colab notebook that records & renders video. https://colab.research.google.com/drive/12osEZByXOlGy8J-MSpkl3faObhzPGIrB

enjoy :)