Pyspark - python path is not recognized

Question

I followed the instruction for installing pyspark on Windows, but am having a similar issue to How to troubleshoot 'pyspark' is not recognized... error on Windows? But my error message when I type 'pyspark' on cmd is:

'C:\Users\simon\AppData\Local\Programs\Python\Python310' is not recognized as an internal or external command, operable program or batch file

I've tried installing different python versions (and edited environment variables) but result is the same. Environment variables are set as fllows:

PYSPARK_PYTHON=C:\Users\simon\AppData\Local\Programs\Python\Python310
SPARK_HOME=C:\apps\spark-3.3.0-bin-hadoop3
HADOOP_HOME=C:\apps\spark-3.3.0-bin-hadoop3

Can you help track down this error?

Solved: adding \python.exe to PYSPARK_PYTHON solved the problem. It is very strange because every tutorial I saw did not include the .exe but oh well!

As @ManojTolagekar said, check if you have installed Python in the `appdata` folder — Olav, Sep 21 '22 at 06:49
check the path for misspellings. or try to copy the path using `shift` + `right click` -> `copy as path` — samkart, Sep 21 '22 at 06:50

score 0 · Answer 1 · answered Sep 22 '22 at 01:35

0

Adding \python.exe to PYSPARK_PYTHON solved the problem.

answered Sep 22 '22 at 01:35

gunserrate

1
1

Pyspark - python path is not recognized

1 Answers1