32

Background

I've just got a new M1 mac mini dev machine, and migrated from my old x86 mac using apple's migration assistant.

Doing that also copied over all my conda environments to the new machine (they were all in my home directory)

I installed the latest version of anaconda and anaconda plus all my python code and environments seem to work fine (this includes a bunch of wheel modules, notably numpy/scipy).

I did a bunch of googling for my questions below, but couldn't find any good answers anywhere - so I thought I'd ask SO as this seems like a quite common situation others will run into

Questions

  • Does anyone know the status of M1 native versions of python/numpy/scipy etc provided by conda forge?
  • I presume that all the binaries in my environments for python/numpy etc all still the old x86 versions, as they were all in environments in my home directory, and running via emulation. So, how do you go about changing/updating those to a M1 arm native version if/when available?
Richard
  • 3,024
  • 2
  • 17
  • 40
  • Does this answer your question? [How to specify the architecture or platform for a new conda environment? (Apple Silicon)](https://stackoverflow.com/questions/65415996/how-to-specify-the-architecture-or-platform-for-a-new-conda-environment-apple) – isuruf Jan 04 '21 at 07:08

6 Answers6

35

A quick update as of July 2021.

TLDR

  • The conda-forge group have a M1 native conda installer here.
  • Installation is simple - run the installer, and you have conda up and running.
  • This will install an M1 native conda, and that conda's default environment will by default install M1 native python versions and M1 native versions of modules (if available).
  • There seem to be native osx M1 native wheels for most common modules now available on the conda-forge channel.

Current status

It seems Anaconda still do not have a native M1 version, nor does Miniconda. ...I can't figure out why it's taken so long and neither still seem to have native M1 support, but that's a separate issue.

Alternative

However, as steff above mentioned, conda-forge (as in the group responsible for maintaining the conda-forge channel) do have a installer for their version of conda that is itself both native M1, and also sets up your environment to pull M1 native wheels where available. This they call Miniforge.

Their github is here.

Various installers for their Miniforge (via direct download, curl or homebrew) can be found on their github page (above) - the direct link to the ARM native miniforge installer is here.

A quick search on conda-forge show's almost all common modules do now have native M1 wheels available. (look for supporting platform 'osx-arm64` eg numpy)

Caveats

I've not tested this too extensively yet, and I'm not sure exactly what happens if a non-M1 wheel is available (I believe it will default to downloading a no-arch version).

I'm also not sure/haven't tested whether you can mix and match M1 wheels with x86 mac wheels. (I'm guessing this would work, but haven't tried).

I also have only done minimal testing using the conda's pip, and how well it recognizes/tries to download/resolves M1 vs x86 pip packages.

Richard
  • 3,024
  • 2
  • 17
  • 40
  • "I believe it will default to downloading a no-arch version." Huh? Most packages are either arch-specific or no-arch, not both. – Stuart Berg Aug 08 '21 at 19:22
  • I doubt that wheels for x86 and M1 could be mixed. If they contain any compiled code, then they couldn't be used together. [Apple's docs say](https://developer.apple.com/documentation/apple-silicon/about-the-rosetta-translation-environment): "The system prevents you from mixing arm64 code and x86_64 code in the same process." – Stuart Berg Aug 08 '21 at 19:24
  • For context on what it took for the conda-forge team to get a build working, [read this page.](https://conda-forge.org/blog/posts/2020-10-29-macos-arm64) Sounds like it was a heroic effort, but they had to use some non-standard tricks to get it all to work. – Stuart Berg Aug 08 '21 at 19:29
  • If you install the arm64 (M1) version, then it will install either arm64 wheels, or failing that noarch. – Richard Aug 13 '21 at 18:16
  • 1
    Yes - I confirmed on the conda forge github site, you cannot run both arm64 and x86 modules at the same time. However, the conda-forge install _does_ let you install both x86 _and_ arm64 environments at the same time, and you can swap back and forth when you need to. – Richard Aug 13 '21 at 18:18
  • Has there been any recent developments in M1 ? Are scikit, tensorflow and keras natively available on M1? – Phani Teja Aug 14 '21 at 19:37
  • not sure on TF and keras, but most of just about anything else is native if you conda-forge. ...I'd guess both of the other two are now as well. – Richard Aug 14 '21 at 21:30
  • @StuartBerg Apple's compiler and linker have always allowed you to have multiple architectures in one binary see the lipo command . What the docs mean is that Intel code can't call Arm code and vice versa. When the object is loaded the system chooses one architecture out of those in the binary – mmmmmm Nov 05 '21 at 15:55
  • @Richard you mentioned that "the conda-forge install does let you install both x86 and arm64 environments at the same time". How does one do this? Is there a way to specify the arch when creating each env? – avivr Dec 19 '21 at 22:53
  • 1
    @avivr Sorry for the late reply. It's not too hard, but it isn't really well documented anywhere. I based mine on this: https://github.com/Haydnspass/miniforge#rosetta-on-mac-with-apple-silicon-hardware. Basically you need to set your CONDA_SUBDIR variable before creating a new environment. In my .bashrc I have the following two aliases set to do that: `alias conda-arm='conda env config vars set CONDA_SUBDIR=osx-arm64'` and `alias conda-x86='conda env config vars set CONDA_SUBDIR=osx-64'` – Richard Dec 29 '21 at 23:45
  • @avlvr So for example, with those aliases, to create/use an x86 env I would a) conda create new_env, b) conda activate new_env c) conda_x86 d) re-install/install python and packages – Richard Dec 29 '21 at 23:47
16

The answer here is going to evolve over time, so here is the most up-to-date knowledge I have as of 27 Jan 2021.

Installing conda in emulation mode works completely fine. All you need to do is to install it in a Terminal run in emulation mode, or else install it using a Terminal emulator that has not been ported over yet.

Once your conda environments are up and running, everything else looks and feels like it did on x86 Macs.

If you'd like a bit more detail, I blogged about my experience. Hopefully it helps you here.

ericmjl
  • 13,541
  • 12
  • 51
  • 80
  • Thanks - I ended up just going with the emulation route you did, as I didn't want to deal with potential compatibility issues of a non-ananconda miniconda like you ran into. It all works fine for me right now, except I don't see anyway yet to see if python or modules are ARM or x86. – Richard Jan 28 '21 at 04:13
  • My pleasure. To respond to your follow-up point, most Python modules are pure Python, so they will only require the Python interpreter to work, and the backing ARM/x86 architecture won't matter. The only places where it does matter is if there are C extensions that are needed, I think. (JAX, as I mentioned in my blog post, is one example.) – ericmjl Jan 29 '21 at 22:44
  • A lot of the modules I use that are the bottlenecks are compiled ...numpy, scipy etc - so it would be great to get native versions of those - but even better have conda check the arch automatically and install the appropriate ones for you – Richard Jan 30 '21 at 21:40
  • The blog post was extremely useful. As a long-time user of MacPorts, it's going to hurt letting go, but it's time to try mini condo, and I will probably still use MP for things like `wget`, etc. – John Laudun Feb 04 '21 at 19:31
  • 2
    I will be instaling miniconda for my Macbook M1. What is emulation mode? What's wrong if I just followed miniconda website? https://docs.conda.io/en/latest/miniconda.html – user13985 Feb 06 '21 at 21:39
  • TensorFlow (except apple version of 2.4) does not work. It's the only library or app I've had trouble with so far. Importing gives an "illegal hardware instruction" error. Based on the solutions I'm seeing, "tensorflow<2.4" may not work on apple m1 anytime soon. – mathandy Feb 07 '21 at 06:04
  • @user13985 I believe ericmjl means running the x86 version on the M1 using Apple's Rosetta "emulator". See [apple.com](https://support.apple.com/en-us/HT211861) for details. – stephinity Mar 19 '21 at 12:57
  • @stephinity that’s what I think too. Then, it’s not really taking an advantage of the M1 chip – user13985 Mar 24 '21 at 17:46
8

I got my M1 about 2 weeks ago and managed to install absolutely everything I need natively from conda-forge and pip. The installer you can download here. As of 5Feb Homebrew is also officially supported on osx-arm64.

steff
  • 896
  • 2
  • 9
  • 23
3

2022/03/02 answers
Native M1 installations are pretty simple now. Here are a few options for Miniforge and Miniconda.

(1) Using Apple's instructions for Tensorflow with Miniforge
This uses the same Miniforge solution mentioned above but includes an M1-optimized Tensorflow install, meaning TF has access to the M1 GPU cores.

Look for the "arm64: Apple Silicon" section at:
https://developer.apple.com/metal/tensorflow-plugin/

(2) Running native M1 with Miniforge and Rosetta with Miniconda side-by-side (Jeff Heaton's tutorial from 2021/11)
Jeff basically uses Apple's solution above for the native Miniforge install.

https://www.youtube.com/watch?v=w2qlou7n7MA

(3) Using native M1 Miniconda There was a native M1 Miniconda installer published in 2021/11: Miniconda3 macOS Apple M1 64-bit bash (Py38 conda 4.10.1 2021-11-08)

https://docs.conda.io/en/latest/miniconda.html

My Experiences
I successfully ran the side-by-side installation from Jeff's tutorial with a few changes. It was very easy and I verified that in the native M1 Miniforge environment that Numpy is using the optimized BLAS/LAPACK linear algebra libraries and that Tensorflow has GPU access. I will update here after I run the Miniconda native M1 installer.

andrew
  • 3,929
  • 1
  • 25
  • 38
1

I installed the native version of python3 through miniforge (Apple version) and Spyder (Intel version) through homebrew and everything is working just fine for me with one exception, I've observed one strange behaviour when setting the "graphics backend" option to "automatic" instead of "inline".

Spyder >>> Prefernces >>> IPython Console >>> Graphics >>> Graphics Backend >>> inline, or automatic

When I start Spyder with the "inline" option and switch to "automatic", the opened kernels function just as expected. However, if I open new consoles they don't work at all. The issue also persists after restarting Spyder. The only way I manage to plot graphics in a separate window is to start Spyder with IPython console "graphics backend" set to "inline" and then change it to "automatic".

If I run python3 through terminal, plotting graphics works just fine as well.

My installation commands were:

brew install --cask miniforge
conda init zsh
conda activate
brew install --cask spyder
brew install PyQt@5
pip3 install matplotlib
Lucas Silveira
  • 43
  • 5
  • As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Dec 03 '21 at 17:36
  • `brew install --cask miniforge` is what I was looking for. Thank you! – garmoncheg Dec 07 '22 at 08:20
  • 1
    @garmoncheg My comment is outdated. Anaconda has a native apple silicon version already. For me, it works better than the miniforge version. – Lucas Silveira Dec 13 '22 at 14:40
1

You can check out this anouncement by Anaconda. You can now use Anaconda on your M1 MAC direclty.

"The 2022.05 release of Anaconda Distribution features native compiling for Apple M1’s ARM64 architecture (boasting 20% faster compute), Anaconda Navigator 2.1.4, conda 4.12.0, as well as several new and updated packages. 2022.05 is also the last release that will support win32."

Daud Ahmed
  • 190
  • 1
  • 10