0

For example:

df = spark.read.json("path")
print(df.schema)

prints:

StructType(List(StructField(timestamp,StringType,true)))

rather than:

StructType([StructField("timestamp",StringType(),True)])

This is an issue for me if i want to come up with a schema by initially inferring the schema from a file in order to then print the schema and hardcode it in my code.

Is there a way to print the schema of a dataframe and have it in python syntax so that i can set a hardcoded schema to a variable in my code and use it?

BubbleTea
  • 1
  • 2

1 Answers1

0

ideally (schema = df.schema) works in case of common file formats like csv etc, but for a file like json it's good to provide schema manually to avoid any error

dsk
  • 1,863
  • 2
  • 10
  • 13