0

I installed Spark 2.4.6 using homebrew on macOS Big Sur. My terminal is iTerm2. I am using .zshrc on a work laptop.

My .zshrc file looks like this:

export SPARK_HOME="/usr/local/Cellar/apache-spark@2.4.6/2.4.6/libexec/bin"
export PATH=~workspace/bin:$SPARK_HOME:$PATH

When I run any of the spark commands listed in the /bin file, for example "spark-shell", I get the following error with an extra "/bin" on the path:

/usr/local/Cellar/apache-spark@2.4.6/2.4.6/libexec/bin: line 60: 
/usr/local/Cellar/apache-spark@2.4.6/2.4.6/libexec/bin/bin/spark-submit: No such file or directory

Some other stackoverflow posts suggested taking off the /bin directory. If I change the .zshrc file to either of the following options, I get a "zsh: command not found" error:

/usr/local/Cellar/apache-spark@2.4.6/2.4.6/libexec
/usr/local/Cellar/apache-spark@2.4.6/2.4.6

And by the way, there is a 2.4.6/bin as well as a 2.4.6/libexec/bin. Any help is appreciated.

BrotherJoseph
  • 1
  • I attempted to go to the /etc/paths file and add both variations with and without the /bin, but I got command not found errors for both. When I added them to etc/paths, I removed the SPARK_HOME variable from .zshrc file – BrotherJoseph Dec 10 '21 at 21:04
  • 1
    Check this https://stackoverflow.com/a/49847490/1950846 – Yassine Abdul-Rahman Dec 11 '21 at 17:34

0 Answers0