spark-shell - How to avoid suppressing of elided stack trace (Exceptions)

Question

I am trying to run on of my scala file from spark-shell. This file calling some other jar files which have already been loaded into spark-context

The problem is if something fails, it only prints the part of the stacktrace. Is there any way I can enable whole stacktrace?

myclassn: ClassifyFields : queryDb -> Problems extracting from DB
  at myclass.queryDb(ClassifyFields.java:231)
  at myclass.getColumnsWithKeys(ClassifyFields.java:258)
  ... 78 elided

Try doing a `try/catch` block and inside the `catch` block, write the stacktrace to a file.. — Sparker0i, May 06 '20 at 14:10

Ram Ghadiyaram · Answer 1 · 2020-05-06T14:54:48.717

5

set the below value to zero.

vals.isettings.maxPrintString=0

like the example below

scala> :power
Power mode enabled. :phase is at typer.
import scala.tools.nsc._, intp.global._, definitions._
Try :help or completions for vals._ and power._

scala> vals.isettings.maxPrintString
res0: Int = 800

scala> vals.isettings.maxPrintString=0
vals.isettings.maxPrintString: Int = 0

scala> vals.isettings.maxPrintString
res1: Int = 0

or after exception happened use the below

lastException.printStackTrace

edited May 06 '20 at 14:54

answered May 06 '20 at 14:47

Ram Ghadiyaram

28,239
13
95
121

1

Is there a way to do this from inside a Jupyter Notebook with Spark Scala code? – Omkar Neogi Dec 21 '20 at 07:43
Proposed solution doesn't help :( – VB_ Aug 29 '22 at 15:45
its doest not help, still see the same error stack trace – Eswar Goud Dec 03 '22 at 05:48

spark-shell - How to avoid suppressing of elided stack trace (Exceptions)

1 Answers1