1

I am using PySpark and did not install Hadoop as such. I am getting this warning Hadoop “Unable to load native-hadoop library for your platform” warning.

Do I need to install Hadoop, if so how can I proceed given that I have installed PySpark on my system?

Note that I am a Novice in Hadoop eco-system.

  • Possible duplicate of [Hadoop "Unable to load native-hadoop library for your platform" warning](https://stackoverflow.com/questions/19943766/hadoop-unable-to-load-native-hadoop-library-for-your-platform-warning) – abiratsis Mar 04 '19 at 13:10

1 Answers1

1

This is just a warning, you can ignore it. Spark and pyspark can be used without hadoop.

You can take a loop at this link:

https://community.hortonworks.com/questions/19897/apache-spark-error-unable-to-load-native-hadoop-li.html

Hichem BOUSSETTA
  • 1,791
  • 1
  • 21
  • 27