43

I try to install a new Python version (3.8) using conda.

!wget -O mini.sh https://repo.anaconda.com/miniconda/Miniconda3-py38_4.8.2-Linux-x86_64.sh
!chmod +x mini.sh
!bash ./mini.sh -b -f -p /usr/local

This works fine. I can call !python script.py to run a 3.8 version.

So, I try my luck with installing another jupyter kernel with Python 3.8 kernel.

!conda install -q -y --prefix /usr/local jupyter
!python -m ipykernel install --name "py38" --user

I check that the kernel is installed.

!jupyter kernelspec list

Then I download the notebook down. Open a text editor to change the kernel specification to

"kernelspec": {
  "name": "py38",
  "display_name": "Python 3.8"
}

This is the same trick that works before, with Javascript, Java, and Golang.

I then upload the edited notebook to Google Drive. Open the notebook in Google Colab. It cannot find the py38 kernel, so it use normal python3 kernel. I run all these cell again.

!wget -O mini.sh https://repo.anaconda.com/miniconda/Miniconda3-py38_4.8.2-Linux-x86_64.sh
!chmod +x mini.sh
!bash ./mini.sh -b -f -p /usr/local
!conda install -q -y --prefix /usr/local jupyter
!python -m ipykernel install --name "py38" --user

It install the Python 3.8 kernel like before. I refresh the browser, to let it connect to the new kernel, hoping it to work like JavaScript, Java, Golang kernel before.

It doesn't work. It cannot connect. Here's the notebook

Any help would be appreciated.

Update (Oct 2022)

Using @ngrislain's method, here's a notebook for Python 3.10 (3.11 also available)

korakot
  • 37,818
  • 16
  • 123
  • 144
  • This is how it works with ijavascript https://colab.research.google.com/gist/korakot/22abd6eccac229e9cb9a027b088b50d6/notebook.ipynb – korakot Mar 23 '20 at 02:22
  • This is how it works with Java https://colab.research.google.com/github/vistec-AI/colab/blob/master/ijava.ipynb – korakot Mar 23 '20 at 02:24
  • This is Golang. https://colab.research.google.com/drive/1-6XkA5OhEA6lMW9DvH4_AcXndC7WppJx – korakot Mar 23 '20 at 02:27
  • What about for python 3.9? See issue in kora: https://github.com/korakot/kora/issues/21 – Charlie Parker Oct 07 '21 at 21:09
  • fyi, one of the suggested answers is not working anymore, see this SO to hopefully solve it: https://stackoverflow.com/questions/69487937/python-3-8-with-kora-in-colab-is-not-working-anymore-how-to-fix – Charlie Parker Oct 07 '21 at 21:15
  • 2
    There is a solution that does not require a local kernel or ngrok. See: https://stackoverflow.com/a/71511943/1854249 – ngrislain Mar 17 '22 at 13:01
  • I wouldn't recommend running a third-party shell containing a binary by any means. – finiteautomata Nov 01 '22 at 21:34
  • 1
    @finiteautomata True indeed. I also provide the `construct.yaml` file, so you can create the binary yourself to be safe. Sometimes, people will opt for convenience at any price, so I provide the binary for maximum convenience. – korakot Nov 03 '22 at 03:50
  • 1
    @korakot that's great to know! No hard feelings, my remark is only general and not a doubt of your good faith. – finiteautomata Nov 13 '22 at 15:12
  • Currently it doesnt seem to work - colab gives the error `Unrecognized runtime "py310"; defaulting to "python3" Notebook settings`and switches to python 3.8. – FamousSnake Dec 06 '22 at 12:59
  • That’s correct. You then install py310 runtime then reload to make it works. – korakot Dec 08 '22 at 18:33

8 Answers8

38

I have found how to run Python 3.8 notebook on Colab.

  • install Anaconda3
  • add (fake) google.colab library
  • start jupyterlab
  • access it with ngrok

Here's the code

# install Anaconda3
!wget -qO ac.sh https://repo.anaconda.com/archive/Anaconda3-2020.07-Linux-x86_64.sh 
!bash ./ac.sh -b

# a fake google.colab library
!ln -s /usr/local/lib/python3.7/dist-packages/google \
       /root/anaconda3/lib/python3.8/site-packages/google

# start jupyterlab, which now has Python3 = 3.8
!nohup /root/anaconda3/bin/jupyter-lab --ip=0.0.0.0&

# access through ngrok, click the link
!pip install pyngrok -q
from pyngrok import ngrok
print(ngrok.connect(8888))
korakot
  • 37,818
  • 16
  • 123
  • 144
  • 15
    thats the greatest achievement in the history of mankind :) Thanks so much – Piakkaa Sep 08 '20 at 17:44
  • 1
    but I can't import libraries, `import boto3` gives error module not found – poon gilbert Sep 08 '20 at 18:36
  • did you try appending the python3.6 packages path? `sys.path.append('/usr/local/lib/python3.6/dist-packages')`. Not sure if its the proper way but it worked for me – Piakkaa Sep 08 '20 at 18:52
  • 4
    `!/root/anaconda3/bin/pip install boto3` this works too – Piakkaa Sep 08 '20 at 19:02
  • 4
    `os.environ['PATH'] = '/root/anaconda3/bin:' + os.environ['PATH']` will add the new pip and conda path to your jupyter notebook environment – Piakkaa Sep 08 '20 at 19:11
  • @korakot Just following your recipe, not only can I SSH from my local VS Code into Colab that has Python 3.8 but also the Python 3.8 Jupyter server setting on Colab is ready for use automatically. Thanks for the brilliant solution. I have a question: I tried to modify one step of your recipe by replacing `install Anaconda3` with `install miniconda3` and executing `conda install Jupyter` manually. I found the VS Code no longer detects the Jupyter server. Do you know how to fix this problem? – Li-Pin Juan Sep 19 '20 at 12:24
  • @Li-PinJuan I tried miniconda at first and failed too. That's why I use Anaconda that somehow config Jupyter correctly. Hope that one day, we will find how to use the smaller miniconda. – korakot Sep 19 '20 at 14:07
  • @korakot does this still work? I'm not able to run kernels via Python 3 with this method. I tried running `jupyter-lab` without nohup to inspect error messages — seems something doesn't work anymore: `'google.colab._kernel.Kernel' could not be imported` – janniks Mar 24 '21 at 18:07
  • what about for python 3.9? – Charlie Parker Oct 07 '21 at 20:59
  • 1
    Hi @korakot This doesn't seem to work the way it should I got this thrown at me : The connection to http://44ee-xx-xxx-xx-xxx.ngrok.io/ was successfully tunneled to your ngrok client, but the client failed to establish a connection to the local address [localhost:8000](http://localhost:8000/). Make sure that a web service is running on [localhost:8000](http://localhost:8000/) and that it is a valid address. The error encountered was: dial tcp 127.0.0.1:8000: connect: connection refused – yagna thakkar Feb 14 '22 at 09:38
  • You don't need ngrok actually, see: https://stackoverflow.com/a/71511943/1854249 – ngrislain Mar 17 '22 at 13:02
20

we can also use kora pip library

!pip install kora
import kora.install.py38
korakot
  • 37,818
  • 16
  • 123
  • 144
poon gilbert
  • 368
  • 3
  • 5
  • 2
    not sure if im giving you enough credit @korakot. You are kind enough to make the library – poon gilbert Sep 08 '20 at 18:21
  • Now in kora 0.6.6, it adds `/root/anaconda3/bin` to PATH for every new notebook. You can now use `pip` correctly. – korakot Sep 11 '20 at 06:38
  • 2
    Hey korakot, is there any documentation for using kora? – Andrey Kurnikovs Dec 13 '20 at 18:52
  • 3
    I get `FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.6/dist-packages/google' -> '/root/anaconda3/lib/python3.8/site-packages/google'` – Tobi Akinyemi Apr 11 '21 at 11:29
  • 1
    I am also getting: FileExistsError: [Errno 17] File exists: '/usr/local/lib/python3.6/dist-packages/google' -> '/root/anaconda3/lib/python3.8/site-packages/google' – Alexander Chervov Oct 07 '21 at 15:00
  • what about for python 3.9? – Charlie Parker Oct 07 '21 at 20:59
  • 6
    btw, this didn't actually work for me `JSONDecodeError Traceback (most recent call last) in () 1 get_ipython().system('pip install kora') ----> 2 import kora.install.py38` Error: `/usr/lib/python3.7/json/decoder.py in raw_decode(self, s, idx) 353 obj, end = self.scan_once(s, idx) 354 except StopIteration as err: --> 355 raise JSONDecodeError("Expecting value", s, err.value) from None 356 return obj, end JSONDecodeError: Expecting value: line 1 column 1 (char 0)` – Charlie Parker Oct 07 '21 at 21:11
  • related question si py38 is not working anymore: https://stackoverflow.com/questions/69487937/python-3-8-with-kora-in-colab-is-not-working-anymore-how-to-fix – Charlie Parker Oct 07 '21 at 21:16
  • 4
    Unfortunately, it seems the `kora` library is now defunct and does not work as described above. Alas! It was pretty cool. – Dan Nissenbaum Jan 09 '22 at 04:22
  • How to make it work for python 3.10? Do you think it is possible to make it work with the beta version of Python 3.11 (May-2022)? – Diving May 03 '22 at 15:24
10

For ipykernel to work in a colab notebook you need the google-colab package to be installed. If not it silently fails (you can notice the problem by running !python -m ipykernel_launcher).

Simply add the line !conda install -q -y google-colab -c conda-forge and it should work.

!wget -O mini.sh https://repo.anaconda.com/miniconda/Miniconda3-py38_4.8.2-Linux-x86_64.sh
!chmod +x mini.sh
!bash ./mini.sh -b -f -p /usr/local
!conda install -q -y jupyter
!conda install -q -y google-colab -c conda-forge
!python -m ipykernel install --name "py38" --user

You can test this solution in this fixed notebook

Don't forget to reload the browser page after you installed the kernel (as explained in the original post).

ngrislain
  • 944
  • 12
  • 19
  • This is not working for me. Current version still returns "3.7.12". – marsipan Mar 19 '22 at 12:38
  • 4
    You need to reload the browser page after you installed the kernel. – ngrislain Mar 21 '22 at 21:27
  • 2
    Several packages appear with `ModuleNotFoundError: No module named *package x* ` after pip installing them with this method. zarr, rasterio, rasterio.features, rioxarray as rio, contextily as cx, stackstac to name a few. – HarriS Mar 29 '22 at 11:19
  • 3
    To those who read the answer and not the question (like me): this solution only works with an **edited notebook**. Simply copying and executing this code will install a new python (`!python -V` returns 3.8.1), but **will not** change the notebook's runtime (`import sys; sys.version` returns 3.7.13). – paperskilltrees Apr 02 '22 at 19:15
  • @paperskilltrees yes, I exactly have the same problem. But what does mean *edited notebook*? How to make `import sys; sys.version` equals 3.8 with the code above from @ngrislain? Maybe I need make Colab Pro? – sirjay Apr 25 '22 at 18:10
  • @sirjay Please read the OP's question: > Then I download the notebook down. Open a text editor to change the kernel specification to – paperskilltrees Apr 25 '22 at 20:57
  • Works perfectly. Quick check with `!python3 --version` generates `Python 3.8.1`. – Lamar May 17 '22 at 15:42
  • Does this still work with the edited notebook ? I gave it a shot, to no avail though – Elle Aug 19 '22 at 09:00
  • @Elle No, it doesn't. For now, to run Python 3.10 in a new notebook, check this answer https://stackoverflow.com/a/74243532/9303470 – Ecstasy Nov 01 '22 at 04:50
  • @ngrislain thanks a lot! do you know how to turn on the syntax highlighting? – vak Nov 22 '22 at 10:06
4

Update: Originally answered on 2020-03-29, but this answer is now outdated, see the answers above.

Based on these previous answers*, it seems that Google only supports python 2.7 and python 3.6 right now (as of 2020-03-29). However, if you must use python 3.8 you could connect to a local runtime: https://research.google.com/colaboratory/local-runtimes.html

*Previous answers:

Bill Mei
  • 717
  • 1
  • 10
  • 22
  • 1
    Thanks for the info. Hope someday I can fix the kernel code. At least for now I can run Python 3.8 as a script in Colab. – korakot Mar 30 '20 at 01:50
  • do you mind sharing how you are able to run the 3.8 as a script? – Nic Wanavit May 15 '20 at 04:49
  • Yeah I see how SSH'ing and installing that way, while a hack, can make it possible, albeit Colab will delete the environment every time it reloads. However anything you install appears inside only the direct ssh environment and doesn't appear to make it to the "Kernel" in Colab in the Graphical User Interface, which is what the original question was asking about. More info about SSH: https://imadelhanafi.com/posts/google_colal_server/ – Bill Mei Jul 24 '20 at 15:55
3

Try the following code. None of the above answers worked for me.

!sudo apt-get update -y
!sudo apt-get install python3.9
!sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.7 1
!sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 2

Source.

Jeremy Caney
  • 7,102
  • 69
  • 48
  • 77
Jallow
  • 39
  • 2
  • using `!python -V` will return 3.9 version, yet using `from platform import python_version` will still return 3.7 – Anan Raddad Mar 12 '22 at 07:50
  • 1
    same for me, after suscessfully running the above, I write `!python --version` and it say it is 3.7 (and it gives me a problem due to defining the type of a variable... – Martin Mar 13 '22 at 19:03
  • 1
    I don't see this switching the notebook to 3.9 .... you will not get 3.9 in the output of `sys.version` after this would you? which means that your code is not interpreted using a 3.9 interpreter after installing python this way in a colab session. Is there a missing config or step or is it a dead end? – matanster Mar 18 '22 at 10:30
1

Update: Since Apr 27, 2023 Colab's default runtime is Python 3.10.11.

Jaroslav Langer
  • 31
  • 1
  • 4
0

I wasn't able to get any of the above answers to work for me. So I initially installed conda on colab and through conda I installed python 3.8.

Steps:

  1. Install conda (Source)

    !wget https://repo.anaconda.com/miniconda/Miniconda3-py37_4.8.2-Linux-x86_64.sh

!chmod +x Miniconda3-py37_4.8.2-Linux-x86_64.sh

!bash ./Miniconda3-py37_4.8.2-Linux-x86_64.sh -b -f -p /usr/local import sys

sys.path.append('/usr/local/lib/python3.7/site-packages/')

  2. Install python 3.8

    !conda install python=3.8

  3. Check the python version

    !python --version
Viktor Ivliiev
  • 1,015
  • 4
  • 14
  • 21
Darius Singh
  • 9
  • 2
  • It hangs on solving environment: `Collecting package metadata ... Solving environment: failed with initial frozen solve. Retrying with flexible solve. Solving environment: done Collecting package metadata (repodata.json): done Solving environment: | ` It might have to do with the fact that I did the previous `!sudo ... install ... ` answer... but I don't really know - would love to if anyone does... – Martin Mar 13 '22 at 19:17
-6

I am a senior Japanese hacker. I referred to the content of this post because I can't use the assignment expression. We appreciate. However, I had to install the rich Python packages on Colab with conda each time. In fact Colab does not utilize anaconda. So, I pip installed the package on py37's pip list with py38. Create hundreds of Colab packages with Python3.8 pip, read the archived file > decompress it and install the Python3.8 interpreter, dist-package (/usr/local/lib/python3.8/dist -packages), I got 3.8 for both Python in Shell and sys.version in Colab, and I was able to use the default packages. I've documented it on the GitHub page below. I posted it if some kind of Japanese quality procedure would be helpful. The relevant page is below. https://github.com/engbJapan/Programming/blob/main/Colab2py38/filelocal43ColabStoragePageTrypy38.ipynb If you have any questions or concerns, please leave a comment.

engbjapan
  • 1