0

I was trying to test the installation of my ultimate-utils library and I get this bizzare error:

(meta_learning) brandomiranda~/ultimate-utils ❯ pip install -e .
Obtaining file:///Users/brandomiranda/ultimate-utils
  Preparing metadata (setup.py) ... done
Collecting dill
  Using cached dill-0.3.4-py2.py3-none-any.whl (86 kB)
Collecting networkx>=2.5
  Using cached networkx-2.6.3-py3-none-any.whl (1.9 MB)
Requirement already satisfied: scipy in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (from ultimate-utils==0.5.3) (1.7.3)
Collecting scikit-learn
  Using cached scikit_learn-1.0.1-cp38-cp38-macosx_10_13_x86_64.whl (7.9 MB)
Collecting lark-parser
  Using cached lark_parser-0.12.0-py2.py3-none-any.whl (103 kB)
Collecting torchtext==0.10.1
  Using cached torchtext-0.10.1-cp38-cp38-macosx_10_9_x86_64.whl (1.6 MB)
Collecting tensorboard
  Using cached tensorboard-2.7.0-py3-none-any.whl (5.8 MB)
Collecting pandas
  Using cached pandas-1.3.4-cp38-cp38-macosx_10_9_x86_64.whl (11.4 MB)
Collecting progressbar2
  Using cached progressbar2-3.55.0-py2.py3-none-any.whl (26 kB)
Collecting transformers
  Using cached transformers-4.13.0-py3-none-any.whl (3.3 MB)
Requirement already satisfied: requests in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (from ultimate-utils==0.5.3) (2.26.0)
Collecting aiohttp
  Using cached aiohttp-3.8.1-cp38-cp38-macosx_10_9_x86_64.whl (574 kB)
Requirement already satisfied: numpy in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (from ultimate-utils==0.5.3) (1.21.4)
Collecting plotly
  Using cached plotly-5.4.0-py2.py3-none-any.whl (25.3 MB)
Collecting wandb
  Using cached wandb-0.12.7-py2.py3-none-any.whl (1.7 MB)
Collecting matplotlib
  Using cached matplotlib-3.5.0-cp38-cp38-macosx_10_9_x86_64.whl (7.3 MB)
Requirement already satisfied: torch<1.10.0,>=1.4.0 in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (from ultimate-utils==0.5.3) (1.9.1)
Requirement already satisfied: torchvision<0.11.0,>=0.5.0 in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (from ultimate-utils==0.5.3) (0.10.1)
Requirement already satisfied: torchaudio<0.10.0,>=0.9.1 in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (from ultimate-utils==0.5.3) (0.9.1)
ERROR: Package 'ultimate-utils' requires a different Python: 3.8.1 not in '>=3.9.0'

however, if you check the dependencies seem to have installed correctly - so I am puzzled what this message means.

In addition if you read the message more carefully you also see that it references 3.8 for some reason (python 3.8? not sure), see:

Requirement already satisfied: torch<1.10.0,>=1.4.0 in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (from ultimate-utils==0.5.3) (1.9.1)
Requirement already satisfied: torchvision<0.11.0,>=0.5.0 in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (from ultimate-utils==0.5.3) (0.10.1)
Requirement already satisfied: torchaudio<0.10.0,>=0.9.1 in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages (from ultimate-utils==0.5.3) (0.9.1)
ERROR: Package 'ultimate-utils' requires a different Python: 3.8.1 not in '>=3.9.0'

but if I ask for pythons version its the one I expected 3.9:

(meta_learning) brandomiranda~/ultimate-utils ❯ python
Python 3.9.7 (default, Sep 16 2021, 08:50:36)
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

So I am very puzzled what is going on. I wonder if pycharm has anything to do with this. But why is it saying it did not install my uutils library?

and checking the pip list confirms its not there:

(meta_learning) brandomiranda~/ultimate-utils ❯ pip list
Package                Version
---------------------- ---------
argcomplete            1.12.3
astroid                2.3.3
boto                   2.49.0
certifi                2021.10.8
cffi                   1.15.0
charset-normalizer     2.0.9
cloudpickle            2.0.0
crcmod                 1.7
cryptography           36.0.0
fasteners              0.16.3
gcs-oauth2-boto-plugin 3.0
google-apitools        0.5.32
google-reauth          0.1.1
gsutil                 5.5
gym                    0.21.0
httplib2               0.20.2
idna                   3.3
isort                  4.3.21
lazy-object-proxy      1.4.3
learn2learn            0.1.6
mccabe                 0.6.1
monotonic              1.6
nodejs                 0.1.1
numpy                  1.21.4
oauth2client           4.1.3
optional-django        0.1.0
Pillow                 8.4.0
pip                    21.3.1
pyasn1                 0.4.8
pyasn1-modules         0.2.8
pycparser              2.21
pylint                 2.4.4
pyOpenSSL              21.0.0
pyparsing              3.0.6
pyu2f                  0.1.5
qpth                   0.0.15
requests               2.26.0
retry-decorator        1.1.1
rsa                    4.7.2
scipy                  1.7.3
setuptools             41.2.0
six                    1.14.0
torch                  1.9.1
torchaudio             0.9.1
torchvision            0.10.1
tqdm                   4.62.3
typing_extensions      4.0.1
urllib3                1.26.7
wrapt                  1.11.2

running an import from uutils confirms it was not installed:

(meta_learning) brandomiranda~/ultimate-utils ❯ python -c "import uutils; uutils.hello()"
python -c "import uutils; uutils.torch_uu.hello()"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'uutils'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'uutils'

what is bizzare is that this has worked before in python 3.9. So something changed and I can't figure out what.

it must be a python path issue since now it's opening 2.7 after deleting python 3.8 (which I never installed...). I think conda init is not initiating my shell correctly. Where is:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/Users/brandomiranda/miniconda3/bin/conda' 'shell.zsh' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/Users/brandomiranda/miniconda3/etc/profile.d/conda.sh" ]; then
        . "/Users/brandomiranda/miniconda3/etc/profile.d/conda.sh"
    else
        export PATH="/Users/brandomiranda/miniconda3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<

suppose to go in the .zshrc file?

refs/related links:

Charlie Parker
  • 5,884
  • 57
  • 198
  • 323
  • Did you check if the path to python folder in enviroment variables was to the older version of python or the older version? Try using `python -m pip install ` – D4NieLDev Dec 10 '21 at 16:36
  • @D4NieLDev funny, that is what I am checking right now. To be more precise I think conda init did not initialize my shell correctly. Where is the conda init stuff suppose to get? (see edit of question) – Charlie Parker Dec 10 '21 at 16:38
  • 1
    I dont really know about where conda files should be, but if you say your python version on `python -V` >= 3.9.0 `python -m pip install ` should work. – D4NieLDev Dec 10 '21 at 16:41
  • @D4NieLDev my path is `(meta_learning) brandomiranda~ ❯ echo $PATH /usr/local/opt/openssl@1.1/bin:/Library/Frameworks/Python.framework/Versions/3.8/bin:/Users/brandomiranda/.pyenv/shims:/Users/brandomiranda/.pyenv/bin:/Users/brandomiranda/anaconda3/bin:/Users/brandomiranda/.opam/4.07.0/bin:/Users/brandomiranda/.gem/ruby/2.6.0/bin:/usr/local/opt/ruby/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/opt/openssl@1.1/bin:/Users/brandomiranda/miniconda3/envs/meta_learning/bin:/Users/brandomiranda/miniconda3/condabin:/Users/brandomiranda/bin#:/Users/brandomiranda/bin/Maude-2.7.1-osx` – Charlie Parker Dec 10 '21 at 16:44
  • which obviously looks wrong. But it should not be like that in the first place. Something is not editing my python paths correctly it seems. Most likely conda. – Charlie Parker Dec 10 '21 at 16:44
  • I am going to try to re-install anaconda and then install the graphical version since that one might not require me to google obscure things about how to remove conda, see details here https://stackoverflow.com/questions/42182706/how-to-uninstall-anaconda-completely-from-macos – Charlie Parker Dec 10 '21 at 16:54
  • removing miniconda: https://stackoverflow.com/questions/29596350/how-to-uninstall-mini-conda-python – Charlie Parker Dec 10 '21 at 16:55
  • Did you check `which pip`? It looks to me like `pip` is resolved to a different, Python 3.8, environment. – merv Dec 10 '21 at 17:02
  • 1
    @merv yes that is correct. It seems my `PATH` is messed up. But somehow after running initializing my shell my `.zshrc` somehow isn't sufficient or correct to have the right `PATH`. I suspect something odd with conda might be going on too. So I am re-installing conda from scratch and seeing if that fixes it. If you have an idea on how the `PATH` should be set up let me know, I've never had so many issues with this before... – Charlie Parker Dec 10 '21 at 17:04
  • PATH manipulation issues often are not a Conda problem, but the terminal (i.e., what interfaces the shell). Is it also broken in Mac Terminal? or just in PyCharm? – merv Dec 10 '21 at 17:18
  • 1
    @merv it is borken in mac terminal. I don't know what the issue is. My `.zshenv` doesn't have anything in it except sourcing `.zshrc`. I am not sure what to do. I will start by trying that my `.zshrc` file starts with a brand new `PATH`. e.g. `PATH=/usr/bin:/bin:/usr/sbin:/sbin`then do `export PATH` in the `.zshrc` file. Seems hacky but I need to start somewhere. – Charlie Parker Dec 10 '21 at 17:22
  • related: https://superuser.com/questions/121870/how-do-i-reset-the-path-variable-on-mac-os-x – Charlie Parker Dec 10 '21 at 17:28
  • 1
    @merv that seems to have solved it. Basically I just overwrite my path with a virgin path. – Charlie Parker Dec 10 '21 at 17:47
  • weird, things seem to work with the terminal but pycharm is broken, I wonder if it's related at all to my hacky way of fixing the python env path. Pycharm broken question: https://stackoverflow.com/questions/21236824/unresolved-reference-issue-in-pycharm – Charlie Parker Dec 13 '21 at 22:06
  • So your question is basically just that you are using 3.9, and you are getting an error that you need >3.8? Is that what this boils down to? This is your own code that's throwing this error, right? Can you show the portion of the code of how you're checking version? Because the error says `Python: 3.8.1 not in '>=3.9.0' ` which makes it seem like you are checking a python version in a string. – nanotek Dec 14 '21 at 18:40
  • This question looks specific to this third party tool. Have you tried asking on their github page? Which of their install instructions did you follow? Their recommended install instructions are 3 lines. Stack overflow questions should be minimal, readable, and reproducible, and not putting bounties on one questions to get attention to other questions – nanotek Dec 14 '21 at 18:50

1 Answers1

0

Once I started using python 3.9 (and not 3.10, probably using the most recent version of python is a bad idea, so I learned to be just 1 behind most version of software to avoid weird bugs) and then I made sure my mac's new default shell zsh has a brand new virgin path:

PATH=/usr/bin:/bin:/usr/sbin:/sbin
export PATH

at the top of it.

# In my 2013 mac this seems to be needed
PATH=/usr/bin:/bin:/usr/sbin:/sbin
export PATH

# let zplug manage itself
zplug "zplug/zplug"

## Plugins for zsh

# new `jump`
zplug "plugins/wd", from:oh-my-zsh

## prompt
## to test do: print -P '%B%F{red}co%F{green}lo%F{blue}rs%f%b'
# ref: http://zsh.sourceforge.net/Doc/Release/Prompt-Expansion.html#Prompt-Expansion
# ref: reference: https://wiki.archlinux.org/index.php/Zsh#Colors
# ref: https://scriptingosx.com/2019/07/moving-to-zsh-06-customizing-the-zsh-prompt/
# %F{color} sets colour until next color of %f, %B infront of that changes tone

PROMPT="%B%F{magenta}%n%B%F{red}%~%B%F{green} %F{cyan}❯%f "


## jump TODO: switch to zsh's wd

#conda activate metalearning
# conda activate synthesis
conda activate meta_learning

## Aliases
alias grep='grep --color=auto'
alias intel='ssh -i ~/.ssh/intel_id_rsa miranda9@ssh-iam.intel-research.net'
alias hal='ssh miranda9@hal.ncsa.illinois.edu'
alias vision='ssh miranda9@vision-submit.cs.illinois.edu'
alias nano='ssh miranda9@nano.ncsa.illinois.edu'
alias dgx='ssh miranda9@hal-dgx.ncsa.illinois.edu'
alias sshcc='ssh miranda9@cc-login.campuscluster.illinois.edu'

#alias tb='tensorboard --logdir'
alias tbb='sh /Users/brando/ultimate-utils/run_tb.sh'

export WANDB_API_KEY=MY SCRETE MUAHAHAHA

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/Users/brandomiranda/opt/anaconda3/bin/conda' 'shell.zsh' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/Users/brandomiranda/opt/anaconda3/etc/profile.d/conda.sh" ]; then
        . "/Users/brandomiranda/opt/anaconda3/etc/profile.d/conda.sh"
    else
        export PATH="/Users/brandomiranda/opt/anaconda3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<

I also reinstalled python, removed python framework 3.8 and reinstalled conda as a graphical interface but I don't think these were the things that made things work. I am fairly confident that is right.

Charlie Parker
  • 5,884
  • 57
  • 198
  • 323
  • 1
    Does this solve your problem, or is this more info to your question? Since you've answered your own question, from an onlooker it seems your question is resolved – nanotek Dec 14 '21 at 18:35
  • 1
    @nanotek indeed this seems to have resolved my question (for now). But I do prefer to accept other's answers than my own answer to my question. Usually they are better. Especially in this case where I don't feel I understand the PATH issues/nuances in python and it feels more of a hack that is currently working. – Charlie Parker Dec 14 '21 at 21:13