2

In Databricks SQL editor , I am trying to get the SUM of few 'Bigint' and 'Double' data type columns of a table. But am getting below error.

Job aborted due to stage failure: Task 0 in stage 29.0 failed 4 times, most recent 
failure: Lost task 0.3 in stage 29.0 (TID 2517) (10.128.2.66 executor 3): 
org.apache.spark.SparkArithmeticException: [ARITHMETIC_OVERFLOW] long overflow. If 
necessary set ansi_mode to "false" to bypass this error.

How to set ansi_mode from Databricks SQL editor?

I tried below in SQL editor

SET spark.sql.ansi.enabled = false

Getting below error

Error running query: org.apache.spark.sql.AnalysisException: Configuration 
spark.sql.ansi.enabled is not available.

How to resolve this Arithmetic overflow error in Databricks?

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
kiruba
  • 129
  • 5

1 Answers1

3

It should be just:

set ansi_mode = false;

See documentation for supported configurations, and examples for set command.

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
  • Thanks Alex. I updated the ansi_mode value as false through the SET command But the value is not getting updated . It still shows true for ansi_mode. Any suggestions? – kiruba Sep 27 '22 at 06:04
  • 1
    just rechecked - works for me... – Alex Ott Sep 27 '22 at 07:06