0

This is different from How to create an empty DataFrame with a specified schema?

The difference is I am asking "without SparkContext". The reason I ask this requirement is

Since Spark 2.x.x, All three data abstractions are unified and SparkSession is the unified entry point of Spark.

If they are all unified, can we create an empty DataFrame with specified schema without SparkContext (with SparkSession only)

Basically, I am looking for something like SparkSession.createDataFrame.

Jill Clover
  • 2,168
  • 7
  • 31
  • 51
  • 2
    The first two answers in the linked question seems to cover some of the ways to do it. For example `spark.emptyDataset[Person]` won't use `SparkContext`. (by the way, `SparkContext` is accessible through `SparkSession` by using `spark.sparkContext`.) – Shaido May 09 '18 at 10:00
  • @shaido Yes, it is accessible, but creatDataframe is not accessible in `spark.sparkContext` – Jill Clover May 09 '18 at 10:44
  • @JohnHass it should be accessable through `spark.createDataframe`. – Shaido May 09 '18 at 10:47
  • @Shaido I ask this question, because `spark.createDataframe` does not provide `createDataframe`. – Jill Clover May 09 '18 at 11:11

0 Answers0