1

I am loading data from postgresql database to spark dataframes using spark-JDBC connection.

I am able to read PSQL tables data by establishing connection but when trying to view entire table using

tweetsDF = sqlContext.sql("SELECT * FROM twitter_tweets").show()

I am facing UnicodeEncodeError

  File "/home/jmeruga/Documents/SPARK/spark/examples/src/main/python/sql/datasource1.py", line 33, in <module>
tweetsDF = sqlContext.sql("SELECT * FROM twitter_tweets").show()
File "/usr/local/spark/spark-2.2.0-bin-hadoop2.7/python/lib/pyspark.zip/pyspark/sql/dataframe.py", line 336, in show

UnicodeEncodeError: 'ascii' codec can't encode characters in position 12476-12481: ordinal not in range(128)

I am getting above error when value of that field is more.

Can anyone suggest me how to resolve this in pyspark?

Yaron
  • 10,166
  • 9
  • 45
  • 65
Jayasree
  • 93
  • 1
  • 7

0 Answers0