-2

I am trying to load the table from MySQL db using pyspark. I have written the below code:

from  pyspark.sql import SparkSession
from pyspark.sql import SQLContext

hostname='localhost'
jdbcPort=3306
dbname='db'
username='user'
password='password'

#jdbc_url = "jdbc:mysql://{0}:{1}/{2}".format(hostname, jdbcPort, dbname)
url="jdbc:mysql://"
# For SQLServer, pass in the "driver" option
# driverClass = "com.microsoft.sqlserver.jdbc.SQLServerDriver"
# Add "driver" : driverClass
connectionProperties = {
  "user" : username,
  "password" : password
}
pushdown_query = "select * from table LIMIT 10;"
df = spark.read.jdbc(url=url, dbtable=pushdown_query, properties=connectionProperties)
#sqlContext=SQLContext(sc)
#df=sqlContext.read.jdbc(url=url, table=pushdown_query, properties=properties)
display(df)

But I am getting the below error:

    ---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-21-70890f1cf807> in <module>()
     15 }
     16 pushdown_query = "select * from table LIMIT 10;"
---> 17 df = spark.read.jdbc(url=url, dbtable=pushdown_query, properties=connectionProperties)
     18 #sqlContext=SQLContext(sc)
     19 #df=sqlContext.read.jdbc(url=url, table=pushdown_query, properties=properties)

AttributeError: 'property' object has no attribute 'jdbc'

Can anyone help me with this error?

Thanks

rnvs1116
  • 39
  • 2
  • 12
  • You get an exception because you didn't initialize `SparkSession`. Also the query is incorrect. – zero323 Jan 02 '18 at 13:11
  • `spark = SparkSession.builder.config(conf=SparkConf()).getOrCreate()`. Later you'll have to [fix the query](https://stackoverflow.com/a/34367336/6910411). – zero323 Jan 02 '18 at 19:10
  • Well, please read how to create a [mcve], and follow the procedure. – zero323 Jan 02 '18 at 19:20

1 Answers1

-2

Please try below code for reading the data from mysql.

hostname = ""
dbname = ""
jdbcPort = 
jdbc_url = "jdbc:mysql://{0}:{1}/{2}".format(hostname, jdbcPort, dbname)

connectionProperties = {
  "user" : username,
  "password" : password
}

query = "select * from table_name"
df = spark.read.jdbc(url=jdbc_url, dbtable=query, properties=connectionProperties)
df.show()

Let me know if it helps.

Neeraj Bhadani
  • 2,930
  • 16
  • 26