I'm using Python 3.8 and when I try to run bin\pyspark under my SPARK_HOME directory , I'm getting the following error :
WARN ProcfsMetricsGetter: Exception when trying to compute pagesize, as a result reporting of ProcessTree metrics is stopped
I tried every possible solution on Stack Overflow but nothing seems to work . I set my PYTHONPATH , Java Home etc.
Can someone please help me resolve this error ?
I'm using Spark 3.0.1