Duplicate columns in spark read json

Question

I ran spark.read.json(json_file_path) on a json file with a flat structure.

I got the error message:

Reference 'col' is ambiguous, could be: col#29582L, col#29632

It is possible that this is due to another column with a name of different case : Col. How could this be resolved if I cannot modify the json data?

Answer 1

After referring to the R case of the problem and searching for the configurations.

I found

spark.conf.set("spark.sql.caseSensitive", "true")

from https://databricks.com/blog/2016/08/15/how-to-use-sparksession-in-apache-spark-2-0.html