Create an empty DF using schema from another DF (Scala Spark)

Question

I have to compare a DF with another one that is the same schema readed from a specific path, but maybe in that path there are not files so I've thought that I have to compare it with a null DF with the same columns as the original.

So I am trying to create a DF with the schema from another DF that contains a lot of columns but I can't find a solution for this. I have been reading the following posts but no one helps me:

How to create an empty DataFrame with a specified schema?

How to create an empty DataFrame? Why "ValueError: RDD is empty"?

How to create an empty dataFrame in Spark

How can I do it in scala? Or is better take other option?

score 5 · Accepted Answer · answered Mar 19 '21 at 21:11

5

originalDF.limit(0) will return an empty dataframe with the same schema.

answered Mar 19 '21 at 21:11

krezno

392
1
12

Create an empty DF using schema from another DF (Scala Spark)

1 Answers1