0

Edit: My question is not regarding how to immediately get pip to work. I fixed this by relinking as you can see in the question.

The question is: What is best/pythonic practice on a fresh *NIX system? One would probably want to have the most recent version of both Python2 and especially Python3 with pip and setuptools and so on. A safeguarded virtual environment seems to be the way, but anyway one has to start somewhere.

So what is the recommendation?

From the comments it seems that one should use the get-pip.py script to install and then use pip to install virtualenv. I guess this has to be done both for Python2 and Python3 if one wants to have both.

This is what happened some days ago:

Ubuntu 18.04 and Python 2.7

I am a heavy R user, but use more and more Python. Virtually every time I set up any kind of customized python environment on Linux or OSX, something breaks and has to be fiddled with.... links, binaries, paths, dependencies. Each and every time. How many thousands of people are sitting on a Ubuntu/Debian box right now and do apt install python pip; pip install --upgrade pip and -duh- it breaks? Really?

Specifically right now: I want to install django on a webserver and started with apt install python-pip.

Then I did the recommended pip install --upgrade pip which installed pip-18.0 but gave the message Not uninstalling pip at /usr/lib/python2.7/dist-packages

After that pip --version threw an error

Traceback (most recent call last): File "/usr/bin/pip", line 9, in from pip import main

It turned out that the old pip still resides in /usr/bin/pip while the new one is in /usr/local/bin/pip.

I fixed it brute force with:

rm /usr/bin/pip
ln -s /usr/local/bin/pip /usr/bin/pip

Did I do the right thing or is this the way to doom down the road?

Is there a pythonic or elegant way to handle such issues or to prevent them in the first place?

With best regards and thanks for any kind of suggestions.

Reinhard Seifert
  • 53
  • 1
  • 11
  • 1
    Do not use the `python-pip` package. Install `pip` by other means. It does not make sense to use a package manager to install one version of a package and then do something else to upgrade it. Don't be surprised if this breaks the install. Either you use the package provided by the distro as is, or you handle it "manually", – Giacomo Alzetta Aug 24 '18 at 11:08
  • I totally agree with previous comment. I would add using a virtual environment, and all your problems are gone, you start with a new fresh python installation (this is the python way I assume). – E.Serra Aug 24 '18 at 11:26
  • Also why starting this way with python 2.7? Use some version of python 3, djongo runs perfectly with it. Once you have python installed django installation is just ```pip install Django==2.1```, or ```pip3 install Django==2.1``` – Evgeny Aug 24 '18 at 12:45
  • @GiacomoAlzetta That sounds like a good idea, but how do I install pip without apt? – Reinhard Seifert Aug 24 '18 at 12:57
  • @E.Serra I needed pip in the first place to `pip install virtualenv` following a post on digitalocean.com – Reinhard Seifert Aug 24 '18 at 12:58
  • No you don't: https://virtualenv.pypa.io/en/stable/installation/ – E.Serra Aug 24 '18 at 13:01
  • @EPo ... because I am a bit confused about Python 2.6, 2.7, 3.6 .... `/usr/bin/python` `/usr/bin/python2` `/usr/bin/python3` `/usr/bin/python2.7` `/usr/bin/python3.5` `/usr/local/bin/python` .... sometimes pip2 does not find or install, sometimes pip3 does or does not .... simply statistically Python2 gave me less head-ache than Python3, so I generally run with Python2 although it seems to me that 3 code is more elegant... – Reinhard Seifert Aug 24 '18 at 13:02
  • 2
    Yes that's normal in the beginning, don't worry about your python version too much, one thing at a time. I would suggest installing virtualenv from source then making one virtualenv per project, you install the packages the project needs and nothing else, once you are done you can do pip freeze and all your dependencies magically appear. It's all advantages! – E.Serra Aug 24 '18 at 13:05
  • well, many linux distos ship python 2.7 for compatibity with linux system programs, but there is not reason to use it in your own projects. You can add an alias for python3 as shown here https://askubuntu.com/a/475815 (also discusses environments). – Evgeny Aug 24 '18 at 13:09
  • Thanks for all the comments the most useful probably from @E.Serra After a bit reading on the provided link I understood how profound virtualenv is in connection with handling Python versions and dependencies including pip and setuptools. I really wonder why Pythons is packaged without it? I will now *always* install virtualenv from source before doing anything with Python. – Reinhard Seifert Aug 24 '18 at 13:23
  • You don't need to install it from source, the easiest is with pip but since it is broken that's the easiest way. Also you will eventually need to fix your pip so good luck with that (I would suggest uninstalling and starting again) – E.Serra Aug 24 '18 at 13:24
  • Possible duplicate of [Error after upgrading pip: cannot import name 'main'](https://stackoverflow.com/questions/49836676/error-after-upgrading-pip-cannot-import-name-main) – anthony sottile Aug 26 '18 at 16:25

1 Answers1

0

My conclusion after investigating the hints in the comments is:

  1. Python conflicts with itself.
  2. Do not try to avoid this - cope with it.
  3. Use virtual environments to handle the unavoidable multiple Python versions consistently.
  4. pyenv seems to work best for my purposes. Use the following to install first pyenv (1), update it (2), install a Pythons version (for this example 3.5.6) with pyenv (3), set your user-global python to the installed version (4), upgrade the Python package mangement tools (5) and install anything inside the virtual Python enviroment (6).
    1. curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash
    2. pyenv update
    3. CONFIGURE_OPTS=--enable-shared pyenv install 3.5.6
    4. pyenv global 3.5.6
    5. pip install --upgrade pip setuptools wheel
    6. pip install ipython jupyter snowflake-connector-python seaborn

CONFIGURE_OPTS=--enable-shared in (3) is needed for some python packages which depend on shared libraries. seaborn will not install without it.

Note that the pyenvinstalled versions includes the package manager pip. Now every call python or pip everything references to the correct version.

When referencing python in scripts in MacOS bash and other *NIXes use #!/usr/bin/env python.

It is possible to install pyenv install 2.7.15 the latest python 2.7 and change to it whenever necessary. pyenv shell 2.7.15 switches to Python2.7 only for a particular shell session when needed to run an adhoc script.

Since I use this flow I have no headaches anymore.

The same setup works superfine so far both on a local MacOS and on a remote Ububtu based jupyter notebook server.

Disclaimer: I am not a hardcore Python fullstack developer, but use Python whenever necessary in Data Science pipelines. So for application development in Python this might not be the best solution, but for consistent management of a Data Science environment it seems to work well.

Hope this is helpful.

Reinhard Seifert
  • 53
  • 1
  • 11