3

I need to Install Python3.7 to support Spark 2 on an M1 Mac.

I can get Python3.7 installed using pyenv, but when I try to install any of the data science libs, like numpy, I get failure: ModuleNotFoundError: No module named '_ctypes'

This seems to be related to libffi. Looking through the build logs for pyenv, I'm seeing this error:

.../Python-3.7.10/Modules/_ctypes/libffi_osx/include/fficonfig.h:51:2: error: "Unknown CPU type"
#error "Unknown CPU type"
 ^
1 error generated.

Python build finished successfully!
Failed to build these modules:
_ctypes               _decimal

Has anyone been able to successfully install 3.7 with numpy natively? I'm not looking for answers that involve Rosetta.

kellanburket
  • 12,250
  • 3
  • 46
  • 73
  • 2
    Make sure the pyenv build does in fact include ctypes. I'd imagine it's autodetecting things wrong... – AKX Nov 19 '21 at 16:43
  • Looks like it's failing to build ctypes. will post logs – kellanburket Nov 19 '21 at 16:47
  • 1
    Yeah, based on that error I think Python 3.7 just doesn't support the M1 without some manual patching to make it understand the cpu it's working with... – AKX Nov 19 '21 at 17:01

2 Answers2

0

Use miniconda to install things: see this post.

Miniconda is an open source version of the conda package/environment manager, as an alternative to pyenv + brew.

It will install ARM native versions of python by default, as well as native versions of modules. This definitely include ARM native numpy.

I've quite sucessfully installed native python 3.7-3.10 and native numpy without any problems at all. (In fact ARM native versions of almost every package I use).

With conda you use conda to install packages, not pip.

So to install numpy you do conda install numpy in the environment you're in.

Richard
  • 3,024
  • 2
  • 17
  • 40
  • Not super famiiar with conda, but was able to find an installer through Miniforge. However, it doesn't seem like python3.7 is available: "PackagesNotFoundError: The following packages are not available from current channels: - python=3.7" – kellanburket Nov 19 '21 at 20:07
  • did you try this command? `conda create -n test_env python=3.7`. That should create a new environment (called 'test_env') with python 3.7 as the python version it uses. This works for me, and I checked the **conda-forge** repository (where miniforge conda will go to get packages) has python 3.5 and later available. – Richard Nov 19 '21 at 20:21
  • 'channels' are conda-speak for repositories. by default miniforge conda should look in the conda-forge repository. At a prompt, type 'conda search python`. That should show you it's using the 'conda-forge' channel, and it will then list all the available versions of python you can install – Richard Nov 19 '21 at 20:23
  • 5
    very weird. I'm on the conda-forge channel, but not seeing anything before 3.8 when I search. same issue with the `conda create` command. It can't find 3.7 – kellanburket Nov 19 '21 at 21:14
  • 2
    I've put a significant amount of time into trying to get 3.7 installed and I'm pretty sure it's not possible. – James Parker Mar 03 '22 at 19:25
  • This is strange!! When I search conda-forge I can find 3.7 (actually all versions back to 1.0.1!). – Richard Mar 04 '22 at 19:27
  • Some time later again... Are you sure you installed the native M1 Miniconda3, for which `conda info` says `platform: osx-arm64`? (Not `platform: osx-64` for Intel, which will be using Rosetta2.) For me, `conda search 'python[subdir=osx-arm64]'` and `... -c conda-forge` only find 3.8.x thru 3.10.x today. But for Intel `osx-64` I indeed get 1.0.1 thru 3.10.5, on conda-forge. (For `linux-aarch64` that is 2.7.15 thru 3.10.5 today.) – Arjan Aug 02 '22 at 17:31
0

I had the same issue and the way I could fix it was to first create a conda environment with a new python version and then change the python version inside the activated environment with:

conda install -c conda-forge python=3.7.8

spadel
  • 998
  • 2
  • 16
  • 40