13

Pyspark is installed in my Mac. This error I am getting while I try to run Pyspark from the command line.

I am installing Pyspark using homebrew and following instructions in this blog(https://medium.com/@roshinijohri/spark-with-jupyter-notebook-on-macos-2-0-0-and-higher-c61b971b5007). When I start running Pyspark in the shell I am getting the following error.

    Python 3.7.1 (default, Dec 14 2018, 13:28:58) 
    [Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    Intel MKL FATAL ERROR: Cannot load libmkl_core.dylib.
vinaykp
  • 539
  • 2
  • 5
  • 16

5 Answers5

8

MKL is Intel's "Math Kernel Library", and it basically handles calculations. The new version of Numpy uses MKL by default, and that is likely why this error is showing up.

Try to update numpy,

conda update numpy

or disable MKL by

conda install nomkl

Hope that helped! Reference: https://github.com/BVLC/caffe/issues/3884

sk311
  • 97
  • 1
  • 8
4

First of all, I tried all the solutions above and neither of them solved this issue for me. For the folks that have a similar situation to me, I'm writing the solution that solved this bug below.

In my case, I have an anaconda virtual environment and doing all of the stuff in there.

Instead of removing mkl, or installing nomkl and etc. I installed it again by using the command:

conda install -c anaconda mkl

This solved the issue for me which I'm trying to solve for a day. I hope it also would be useful for you folks too.

msc
  • 91
  • 8
1

enter image description here

Update your mkl package. That should fix this.

Uncaught Exception
  • 2,149
  • 18
  • 25
1

I was also getting the same error. Then I tried to create a virtual environment and then this error went off.

ishaj
  • 91
  • 1
  • 4
0

If you are getting this error after running makemigrations command in Django, it is highly likely that your virtual environment is not activated.

This happens because when the venv is not activated, python starts wondering off to other directories looking for different libraries.

Running source venv/bin/activate (where venv is name of your virtual environment) fixes issue

The Ambidextrous
  • 87
  • 4