2

I'm using PySpark on a Hadoop cluster and I could not find the info about the executor memory model with Python.

I know that the Python memory (spark.python.worker.memory) does not overlap the JVM Heap (spark.executor.memory).

However, does the Python Memory overlap the Executor Memory Overhead or not ?

Thank you very much,

orayer
  • 47
  • 1
  • 4
  • seems to be discussed in https://stackoverflow.com/questions/36594056/how-does-spark-python-worker-memory-relate-to-spark-executor-memory and https://stackoverflow.com/questions/31565497/what-is-spark-python-worker-memory – Khalid Huseynov Oct 19 '21 at 05:19

0 Answers0