I am loading data from postgresql database to spark dataframes using spark-JDBC connection.
I am able to read PSQL tables data by establishing connection but when trying to view entire table using
tweetsDF = sqlContext.sql("SELECT * FROM twitter_tweets").show()
I am facing UnicodeEncodeError
File "/home/jmeruga/Documents/SPARK/spark/examples/src/main/python/sql/datasource1.py", line 33, in <module>
tweetsDF = sqlContext.sql("SELECT * FROM twitter_tweets").show()
File "/usr/local/spark/spark-2.2.0-bin-hadoop2.7/python/lib/pyspark.zip/pyspark/sql/dataframe.py", line 336, in show
UnicodeEncodeError: 'ascii' codec can't encode characters in position 12476-12481: ordinal not in range(128)
I am getting above error when value of that field is more.
Can anyone suggest me how to resolve this in pyspark?