Change python version to 3.x

Question

According to poetry's docs, the proper way to setup a new project is with poetry new poetry-demo, however this creates a project based on the now deprecated python2.7 by creating the following toml file:

[tool.poetry]
name = "poetry-demo"
version = "0.1.0"
description = ""
authors = ["Harsha Goli <harshagoli@gmail.com>"]

[tool.poetry.dependencies]
python = "^2.7"

[tool.poetry.dev-dependencies]
pytest = "^4.6"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

How can I update this to 3.7? Simply changing python = "^2.7" to python = "^3.7" results in the following error when poetry install is run:

[SolverProblemError]
The current project's Python requirement (2.7.17) is not compatible with some of the required packages Python requirement:
  - zipp requires Python >=3.6

Because no versions of pytest match >=4.6,<4.6.9 || >4.6.9,<5.0
 and pytest (4.6.9) depends on importlib-metadata (>=0.12), pytest (>=4.6,<5.0) requires importlib-metadata (>=0.12).
And because no versions of importlib-metadata match >=0.12,<1.5.0 || >1.5.0
 and importlib-metadata (1.5.0) depends on zipp (>=0.5), pytest (>=4.6,<5.0) requires zipp (>=0.5).
Because zipp (3.1.0) requires Python >=3.6
 and no versions of zipp match >=0.5,<3.1.0 || >3.1.0, zipp is forbidden.
Thus, pytest is forbidden.
So, because poetry-demo depends on pytest (^4.6), version solving failed.

Answer 1

Poetry makes it super easy to work with different Python versions or virtual environments. The recommended way to specify your Python version according to Poetry docs is

poetry env use /path/to/preferred/python/version

You can get the path to your Python version by running which python3.7 on Linux or py -0p on Windows.

Answer 2

Use:

poetry env use 3.9

or

poetry env use $(which python3.9)

doesn't work for some reason for python 3.9. Though having explicitly stated version requirement in pyproject.toml file, it's sad that one needs to manually enforce this.

Answer 3

Whenever you change dependencies by hand in your pyproject.toml you have to take care of these points:

1. Run poetry lock --no-update afterwards. The reasons for this is, that poetry install takes the poetry.lock as input if can find one and not the pyproject.toml.

2. If you change the python version and uses in-project virtualenv, remove the .venv before running poetry install. poetry doesn't change the python version of a venv once it is created, because it uses the python version itself to create the virtualenv.

Answer 4

Interestingly, poetry is silently failing due to a missing package the tool itself relies on and continues to install a broken venv. Here's how you fix it.

sudo apt install python3-venv
poetry env remove python3
poetry install

I had to remove pytest, and then reinstall with poetry add pytest.

EDIT: I ran into this issue again when upgrading a project from python3.7 to python3.8 - for this instead of installing python3-venv, you'd want to install python3.8-venv instead

Answer 5

I had the same problem. I solve it by fixing the first line in the file /home/nordman/.poetry/bin/poetry (nordman is my local name).

Just change #!/usr/bin/env python to #!/usr/bin/env python3

Answer 6

You can change in pyproject.toml and execute de this command "poetry env use 3.x" that works for me.

Answer 7

I tried every solution I could find, but could not get poetry to pick up the upgraded version of Python. Finally found something that worked.

tldr;

1. Run poetry config virtualenvs.in-project true to use local venvs
2. Run deactivate to exit any existing venv
3. Run poetry shell to activate a new, local venv

So here's what I was seeing:

(my-project) ~/I/my-project ❯❯❯ poetry env info

Virtualenv
Python:         3.9.6
Implementation: CPython
Path:           /Users/my-user/.local/share/virtualenvs/my-project-some-hash
Valid:          True

System
Platform: darwin
OS:       posix
Python:   /usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9

I tried running poetry env remove python and got an error:

(my-project) ~/I/my-project ❯❯❯ poetry env remove python

  ValueError

  Environment "my-project-some-hash-py3.9" does not exist.

Meanwhile, I saw somewhere that it's recommend to use local virtual environments by setting this property, so I did it:

poetry config virtualenvs.in-project true

This did not solve my problem, but then I realized that changing this setting would not take affect automatically because I was already in another non-local virtual environment (see poetry env info output above).

So based on the docs, I ran deactivate to deactivate the current virtual environment.

Now, I saw this:

~/I/my-project ❯❯❯ poetry env info

Virtualenv
Python:         3.10.1
Implementation: CPython
Path:           NA

System
Platform: darwin
OS:       posix
Python:   /Users/my-user/.pyenv/versions/3.10.1

Now I ran poetry shell to create a new virtual environment and it picked up the new local venvs setting:

~/I/my-project ❯❯❯ poetry shell
Creating virtualenv my-project in /Users/my-user/projects/my-project/.venv
Spawning shell within /Users/my-user/projects/my-project/.venv

And finally, I saw the upgraded Python version I was expecting!

(.venv) ~/I/my-project ❯❯❯ poetry run python -V
Python 3.10.1

Update: After I did all of the above, I noticed that poetry env use works fine!

(.venv) ~/I/my-project ❯❯❯ python -V
Python 3.10.1
(.venv) ~/I/my-project ❯❯❯ poetry env use 3.9.6
Recreating virtualenv my-project in /Users/my-user/projects/my-project/.venv
Using virtualenv: /Users/my-user/projects/my-project/.venv
(.venv) ~/I/my-project ❯❯❯ python -V
Python 3.9.6

Answer 8

I've experienced the exact same behavior on my MacBook! So to make this practical, first, let's see, what is the default Python for the MacBook. I'm running macOS Big Sur 11.2.1 (20D74). In terminal:

python --version                                                                                                                  
Python 2.7.16

With that, let's install poetry. I've used poetry installation script from the GitHub. Using this script is a recommended method to install poetry (though I'd argue this is NOT the best idea to pipe scripts from the Internet into the python interpreter) The installation script is pretty smart: it attempts to detect which python is available on the system:

def _which_python(self):
    """Decides which python executable we'll embed in the launcher script."""
    allowed_executables = ["python", "python3"]
    if WINDOWS:
        allowed_executables += ["py.exe -3", "py.exe -2"]
...

So as the default Python is 2.7.16, and that is what returned by calling python, the entry in ~/.poetry/bin will look like:

#!/usr/bin/env python

And here you go! The default python will end up in the pyproject.toml file and you'll need to do an extra dance to a) make sure python3 is a dependency for your project; and b) that the virtual environment will use python3 as python interpreter.

As @mfalade mentioned, you can set environment with poetry env:

poetry env use /path/to/python3

And also modifying the pyproject.toml with:

...
[tool.poetry.dependencies]
python = "^3.9"
...

From this point you are good to go, poetry will use python3 to create virtual environment for you project, dependency is specified in the file and ... the grass is green again.

But can we use python3 by default? There is an insightful thread on GitHub and another one. And if you look back at the code snippet from the installation script above, you might wonder why not to check for python3 first and then for python, especially taking into account that this is the exact order for Windows installation. Well, you are not alone, I also wonder ;)

I would not suggest editing the ~/.poetry/bin/poetry file directly as this file is generated by the installation script (so what will happen if you run installation script again? And again?).

Really, this is a minor annoyance, and knowing the tool it is easy to work around it. I'd expect that to be mentioned in installation guide though...

Anyway, hope it helps!

Answer 9

You can simply use pyenv for that. Create .python-version file inside your project and poetry will match the exact python version.

# check current python version (set up globally)
❯ pyenv version
3.9.0 (set by /Users/tomasz.zieba@showpad.com/.python-version)

# create .python-version file for project
❯ pyenv local 3.9.0

# check python version again (now it's set up locally)
❯ pyenv version
3.9.0 (set by /Users/tomasz.zieba@showpad.com/Documents/myproject/.python-version)

❯ poetry lock
(...)

❯ poetry run python --version
Python 3.9.0

Answer 10

The way I do it, and it's super effective: Use pyenv to manage the pyenv versions, and it's plugin pyenv-virtualenv to manage the versions as virtual environments, then poetry to pick the version from them automatically as it manages the dependencies:

Use pyenv to install whatever version of python you want to use:

pyenv install -v 3.7

Create a virtual environment based on the installed version 3.7. You will need the plugin pyenv-virtualenv.

pyenv virtualenv 3.7 arshbot_3.7_virtualenv

Assume a project called arshbot_proj. Working in $HOME directory:

mkdir ~/arshbot_proj
cd ~/arshbot_proj

Use the virtual env we just created. Attach this project to it. Below creates .python-version file indicating arshbot_3.7_virtualenv.

pyenv local arshbot_3.7_virtualenv

Use poetry init to create the pyproject.toml inside the dir using arshbot_3.7_virtualenv. You could also use poetry new instead of poetry init to create project structure alongside pyproject.toml instead of making your project first in above step. With pyproject.toml and .python-version both inside this dir, poetry will automatically pick the 3.7 courtesy of arshbot_3.7_virtualenv. Poetry will also use this virtual env to install packages at ~/.pyenv/versions/3.7/envs/arshbot_3.7_virtualenv/lib/python3.7/site-packages

poetry init

That is it. Poetry will pick the 3.7 automatically every time you run it from inside that directory attached to that virtual environment, with it activated. For different python version, just repeat the above steps, replace 3.7 with the new version. The virtual environment will appear twice: In the envs directory as a virtual environment, and also in the versions directory as a version, with contents replicating when poetry installs packages.

BONUS: We try to make it even more clear using a different approach, same concept, same results. Since by default, if you didn't change PYENV_ROOT, pyenv installs every python version in ~/.pyenv/versions: If you want the system version(shipped with your distro) to be part of the versions you can select, use venv to mimick a pyenv installation of a python version, now call it system_ver in place of the 3.7. Since it's already in the system, we don't need pyenv to download it, we copy it over to our versions directory, so that it's available to create a virtual environment

cd ~/.pyenv/versions
python3 -m venv --copies system system_ver
pyenv virtualenv system_ver system_ver_virtualenv

To use it in your project in place of previous version 3.7:

cd ~/arshbot_proj
pyenv local system_ver_virtualenv

Poetry will now use whatever version came originally with your distro. The --copies will ensure venv copies the files instead of use links, so you may omit it. Usually useful if you need to later make a multi stage dockerfile for the project using the files from the virtual environment.

Answer 11

I am running ubuntu 22.04 and only have python 3.10 installed. I've destroyed my system python enough times that I refuse to dance with older versions, or running any installs from pip. What I wanted is an older version of python, that was installed by not installed over my current python install. What I did was follow this guide.

sudo apt install libgdbm-dev build-essential libnss3-dev libreadline-dev libffi-dev libsqlite3-dev libbz2-dev libncurses5-dev libssl-dev zlib1g-dev
cd /tmp
wget https://www.python.org/ftp/python/3.9.14/Python-3.9.14.tgz
tar -xf Python-3.9.14.tgz
cd Python-3.9.14 
./configure --enable-optimizations

## if you want to run in parallel set -j to how many cpus you have
## I subtracted 2 so my machine wouldn't have a stroke
# lscpu | egrep 'CPU\(s\)'
# make -j <cpus you are comfortable with>
make

## Super important to run altinstall to not overwrite
sudo make altinstall

## I updated the pyproject.toml to ^3.9
poetry env use /usr/local/bin/python3.9

Testing this out now, if I don't post anything after this, it either worked or I am not longer living.

Answer 12

Just edit pyproject.toml file and change the version of python to 3.7 as in :

python = "^3.7"