7

I am trying to understand the difference between createOrReplaceGlobalTempView and createOrReplaceTempView.

createOrReplaceTempView is introduced in version 2.0 and createOrReplaceGlobalTempView is introduced in 2.2.

As per the documentation,

for createOrReplaceGlobalTempView : the lifetime of this temporary view is tied to this Spark application.

For createOrReplaceGlobalTempView : The lifetime of this temporary table is tied to the SparkSession that was used to create this DataFrame.

Could you please elaborate the difference on Spark Application and SparkSession.

Regards,

Neeraj

Sushant Yelpale
  • 860
  • 6
  • 19
Neeraj Bhadani
  • 2,930
  • 16
  • 26
  • Please refer to the amazing answer here: [https://stackoverflow.com/questions/42774187/spark-createorreplacetempview-vs-createglobaltempview](https://stackoverflow.com/questions/42774187/spark-createorreplacetempview-vs-createglobaltempview) – human Mar 08 '18 at 05:15

2 Answers2

5

Standard temporary tables live in the user space, similarly to Hive

The data in these tables is stored in the user's scratch directory rather than in the Hive warehouse directory. The scratch directory effectively acts as the user' data sandbox.

and are limited to a single user session.

Global temporary views, are different, in the way that can be accessed by multiple sessions, so there are intermediate between normal view (it can be accessed globally, it is registered in global temporary database) and temporary view (its scope is limited to a single session).

Alper t. Turker
  • 34,230
  • 9
  • 83
  • 115
1

createOrReplaceTempView

When you create view using using createOrReplaceTempView the lifetime of this temporary table is tied to the SparkSession that was used to create this DataFrame. https://spark.apache.org/docs/3.1.3/api/python/reference/api/pyspark.sql.DataFrame.createOrReplaceTempView.html

createOrReplaceGlobalTempView

When you create view using createOrReplaceGlobalTempView the lifetime of this temporary view is tied to this Spark application.

https://spark.apache.org/docs/3.1.3/api/python/reference/api/pyspark.sql.DataFrame.createOrReplaceGlobalTempView.html

RaHuL VeNuGoPaL
  • 447
  • 5
  • 7