I've been a developer for about three years, primarily working in TypeScript and Node.js. I'm trying to broaden my skillset by learning Python (and eventually expanding my learning of computer vision, machine learning (ML), etc.), but I feel incredibly frustrated by trying to get Python to work consistently on my machine. Surely I'm doing something wrong, but I just can't really understand what it is.
I've run into these problems mostly when using ML packages (TensorFlow, Whisper, OpenCV (although I was eventually able to resolve this), so I don't know if it's related to M1 support of one of the common dependencies, etc.
My current understanding of Python is that:
- M1 support of Python is version-dependent at best.
- venv is the only environment manager I should need to use
- I should use pyenv to install Python versions so as to not conflict with OS-installed python (macOS dependencies)
I'll use the latest project I'm working on as an example.
My machine and environment
Mac Pro M1, macOS v12.6 (Monterey)
pyenv 2.3.9
Python 3.7.13
Fish shell version 3.5.1
My general workflow to get a project started:
- Create a virtual environment using venv
python3 -m venv <some_environment_name> - Open created directory using Visual Studio Code, and activate the virtual environment
- Here is where I encounter my first issue, which seems to be persistent.
source /Users/paal/src/whisper_transcription
Output:
/bin/activate.fish
functions: Function '_old_fish_prompt' does not exist
~/src/whisper_transcription/bin/activate.fish (line 18):
functions -c _old_fish_prompt fish_prompt
^
in function 'deactivate' with arguments 'nondestructive'
called on line 30 of file ~/src/whisper_transcription/bin/activate.fish
from sourcing file ~/src/whisper_transcription/bin/activate.fish
(Type 'help functions' for related documentation)
functions: Function 'fish_prompt' does not exist
~/src/whisper_transcription/bin/activate.fish (line 47):
functions -c fish_prompt _old_fish_prompt
^
from sourcing file ~/src/whisper_transcription/bin/activate.fish
(Type 'help functions' for related documentation)
fish: Unknown command: _old_fish_prompt
~/src/whisper_transcription/bin/activate.fish (line 71):
_old_fish_prompt
^
in function 'fish_prompt'
in command substitution
(whisper_transcription)
So, to resolve this, I add the following if statement to the fish.config file.
if type -q $program
_old_fish_prompt
end
Looking at GitHub issues, this seems to be a persistent issue for the Fish shell and this seems to at least temporarily resolve it.
Or, I just switch to Z shell (executable zsh).
OK, so with that resolved I move on. The environment is activated, I'm using Z shell now, and I can successfully run a Python script that prints "hello world" to the console.
Then comes the nightmare of installing any packages. It seems like any project I start has some weird edge case of compatibility issues. Between M1 processors, Python versions, builds not working correctly, etc.
For example,
import whisper
... # The rest of the file
This with any other code or even by itself throws the following error:
Traceback (most recent call last):
File "main.py", line 1, in <module>
import whisper
File "/Users/paal/src/whisper_transcription/lib/python3.7/site-packages/whisper/__init__.py", line 12, in <module>
from .decoding import DecodingOptions, DecodingResult, decode, detect_language
File "/Users/paal/src/whisper_transcription/lib/python3.7/site-packages/whisper/decoding.py", line 514
if prefix := self.options.prefix:
^
SyntaxError: invalid syntax
This appears to be some problem with the Python version. From what I understand, the := operator isn't valid syntax until Python 3.8. However, dependencies of Whisper (PyTorch) only seems to be supported up to version 3.7.9.
So, you can see, it seems like I just end up in these bizarre circular problems where some dependency of some package I want to use isn't supported by either the platform or the current Python version, and they seem basically unsolvable (at least with my current knowledge of Python).
Why is this seemingly so complicated? I'm clearly doing something wrong here, and obviously I'm out of my comfort and knowledge zone, but these issues feel very daunting and opaque, and difficult to actually troubleshoot this in any consistent or clear way.
Is there a resource that makes this stuff more clear? Is Python development on M1 chips just this broken at the moment? How can I get past these seemingly basic issues to start actually learning?
I'm not necessarily looking for a solution to this specific problem here, but if there’s general advice about environment management and how to make things work somewhat reliably, I'm fine troubleshooting. I just feel like every time I start trying to learn, I end up in these rabbit holes that take hours and hours to fix and sometimes don't even really resolve things.
