44

I'm having issues installing OpenAI Gym Atari environment on Windows 10. I have successfully installed and used OpenAI Gym already on the same system.

It keeps tripping up when trying to run a makefile.

I am running the command pip install gym[atari]

Here is the error:

enter image description here

and here is what I currently have on my system...cmake and make are both clearly installed.

enter image description here

dant
  • 687
  • 1
  • 6
  • 15

5 Answers5

156

A while ago I have created a fork with Windows support (devs of original repository do not merge or even comment PRs and issues). It does not require neither MSYS/Cygwin nor CMake or Zlib.

To simply install atari-py wheels (binaries) use this command:

pip install -f https://github.com/Kojoley/atari-py/releases atari_py

If you have any distutils supported compiler you can install from sources:

pip install git+https://github.com/Kojoley/atari-py.git

Test your installation with a simple example:

import gym
env = gym.make('SpaceInvaders-v0')
env.reset()
for _ in range(1000):
    env.step(env.action_space.sample())
    env.render('human')
env.close()  # https://github.com/openai/gym/issues/893

If your got a 'scrambled' output that's most likely because your gym is outdated.

Nikita Kniazev
  • 3,728
  • 2
  • 16
  • 30
  • 12
    This fine piece of work deserves more votes. Quick and issue free way to get gym[atari] working on Win7_64 – Niall Cosgrove Dec 08 '17 at 17:00
  • This is great, worked for me. However, I'm having trouble visualising the emulator, since this installation doesn't have SDL enabled (I guess?). Do you have any advice regarding this? – Daniel Crane Feb 19 '18 at 01:54
  • @DanielCrane I think you doing something wrong. Gym does not use SDL and for me visualization works without issues. I will add a simple usage to the answer. – Nikita Kniazev Feb 19 '18 at 21:17
  • Agree with @NiallCosgrove this is a fine piece of work. Had problems with image rendering but fixed after upgrading to latest gym version as solved in https://stackoverflow.com/a/45150861/2491589 – Marky0 Mar 03 '18 at 16:36
  • 1
    Thank you so much! A simple one line copy/paste saved HOURS of attempts of virtual envs, dependencies, etc that still didn't work, yet this does! – user4779 Apr 26 '19 at 10:07
  • Even in 2020 this is still working. You're the real MVP! – Harpal Jun 08 '20 at 18:06
7

I ended up installing Bash on Ubuntu on Windows and using it to run the OpenAI Gym / Atari environment. OpenAI Gym has very limited support for Windows at the moment.

dant
  • 687
  • 1
  • 6
  • 15
  • I had to manually install zlib for this to work: `sudo apt-get install zlib1g-dev` – Toke Faurby Apr 20 '17 at 08:50
  • Ah yes, there are some stray dependencies that you'll have to pick up, but none of them should prove to be difficult with the simplicity of sudo apt-get install :) – dant Apr 20 '17 at 20:20
  • do you have it running with a display? how are you handling that? xming? – AwokeKnowing May 27 '17 at 04:55
  • @AwokeKnowing, my code didn't involve generating graphics, so I don't have an answer for that unfortunately. – dant Jun 03 '17 at 04:58
  • @dant well I was able to get atari going on gym and pytorch, with graphics and matplotlib plots too, so I'm very happy with ubuntu on windows. If only it had GPU/cuda I would be thrilled. – AwokeKnowing Jun 03 '17 at 05:42
  • @AwokeKnowing, great! Feel free to edit the answer if you want to include what you did! And re: GPU/cuda, it's only the first release so there is hope :) – dant Jun 03 '17 at 06:23
  • @dant I added an answer separately – AwokeKnowing Jun 03 '17 at 16:13
7

This is not fully tested, because I don't remember exactly what I did, but currently I have openAI gym running with all the atari games set up and displaying, and also matplotlib plots, all while using ubuntu on windows (WSL). In fact I have sublimetext3 and spider working too.

So take these as a guide, but I don't have "clean" environment to test them on.

First, in Windows, Google "xming" (x11 server) and download from sourceforge / install / run. This is what makes it all possible.

Now in WSL bash install the display stuff to work with xming

sudo apt-get install x11-apps
export DISPLAY=localhost:0.0 
nano ~/.bashrc  #(add  export DISPLAY=localhost:0.0   at the end. Ctrl+X to exit/save)
sudo apt-get install gnome-calculator #will get you GTK

Now in WSL bash install Anaconda. this will involve downloading the .sh file (eg with curl -O "[the http link to latest anaconda]" and running it with bash [the file].sh. Don't use sudo when installing Anaconda.

With anaconda installed, close WSL, and restart it. Now make an environment and activate it

conda create -n gym python=3.5 anaconda
source activate gym

Now get the gym repo

git clone https://github.com/openai/gym.git
cd gym

Now install these gym dependencies mentioned on openai gym repo

apt-get install -y python-numpy python-dev cmake zlib1g-dev libjpeg-dev xvfb libav-tools xorg-dev python-opengl libboost-all-dev libsdl2-dev swig

Now install libgcc with conda

conda install libgcc

Now build gym

pip install -e '.[all]'

That's basically it. make sure Xming is running on windows, and in WSL type gnome-calculator, and it should bring up the calculator. if it doesn't, keep working on the display side. If it does, try running some of the agents in the gym examples folder.

I may have missed a couple extra dependencies along the way, but these would have been things I figured out based on error messages.

Here's the pic to keep you motivated: enter image description here

EDIT: Today I ran the following command which installed Qt5 as the back end, and matplotlib is working fine with Qt5Agg as the back end (vs TkAgg). This may be helpful if you're running some thing else on WSL which needs Qt5

sudo apt-get update && sudo apt-get install qtbase5-dev

Also, to find your matplotlibrc, and command prompt type: python import matplotlib print(matplotlib.matplotlib_fname()) quit()

Please note that there is NO GPU SUPPORT on ubuntu for windows. This is the top requested feature at uservoice, yet MS has it on "backlog". If you're interested, vote here

AwokeKnowing
  • 7,728
  • 9
  • 36
  • 47
  • 2
    All the scripts seem to be running at an accelerated speed compared to e.g. the OpenAI video demonstrations of the same atari games. Is that caused by using Ubuntu on Windows? Any idea on how to fix that? – Kagaratsch Dec 26 '17 at 18:32
  • Version 1.3.0 of one of the dependencies, pyglet, has a problem which causes a 'NotImplemented: abstract' error when rendering a Gym application in WSL. Uninstalling pyglet and then reinstalling version 1.2.4 fixes the issue. pip install 'pyglet==1.2.4' – DrMcCleod Feb 08 '18 at 12:14
  • @Kagaratsch just use time.sleep(0.1) or something. It useful for machine learning that its so super fast! – Dibran Sep 10 '19 at 09:47
  • 1
    I did a full write up to get this running! https://dibranmulder.github.io/2019/09/06/Running-an-OpenAI-Gym-on-Windows-with-WSL/ – Dibran Sep 10 '19 at 09:48
  • @Dibran very helpful! – AwokeKnowing Sep 17 '19 at 18:59
5

I encountered that gym now requires later version v0.1.4 of atari-py than any of other cloned repos.

Thanks to Nikita Kniazev - I ported his Windows edits to latest code from openai/atari-py and got gym working.

Use: pip install git+https://github.com/Kojoley/atari-py.git

iva2k
  • 450
  • 4
  • 9
1

I had the same issue with the atari-py environment. Then I followed the steps in the Openai GitHub and then it worked.

Zeyd
  • 11
  • 2
  • A shortened version of the relevant information from that link may be useful/improve this answer - just in case the link ever dies or goes out of date. – Gavin Apr 06 '20 at 04:04