0

I am new to Apache Spark. I have a use case where I have to save data frame data in MySQL. I got the below code to do the same:

data_frame.write.format('jdbc').options(
    url='URI',
    driver='com.mysql.jdbc.Driver',
    dbtable=table_name,
    user=user_name,
    password='your_password').mode('append').save()

But when I ran the code, I got the below error:

    File "/usr/local/Cellar/apache-spark/2.3.1/libexec/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py", line 328, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o207.save.
: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver

I might be missing out on very minute detail. How can I fix this?

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Raghav salotra
  • 820
  • 1
  • 11
  • 23

1 Answers1

0

The error description is clearly indicating that it's not able to locate the JDBC driver class. You will have to include the JAR file for com.mysql.jdbc.Driver using

pyspark --jars  <jar-file-location>

See this question - How to add third-party Java JAR files for use in PySpark.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Pushkr
  • 3,591
  • 18
  • 31