1

This works fine on a DF.

val recs = spark.table("IIR").select($"dateFrom").withColumn("N", lit(5))
val rec2 = recs.withColumn("sellByDate", date_add(col("dateFrom"), 5))

However, this does not:

val rec2 = recs.withColumn("sellByDate", date_add(col("dateFrom"), $"N"))

The infamous message is gotten:

notebook:11: error: type mismatch;
  found   : String("$N")
  required: Int

What is the easiest away around this?

thebluephantom
  • 16,458
  • 8
  • 40
  • 83
  • OK, will look but as soon as dates and Cols, a bit of a muddle. – thebluephantom Mar 08 '19 at 17:08
  • 1
    Yeah, the problem is that with support for literals, columns and strings, number of possible signatures grows like crazy. Oh, and it goes without saying that you should ignore the accepted answer and go straight to [`expr`](https://stackoverflow.com/a/44368970/10465355). – 10465355 Mar 08 '19 at 17:09
  • 1
    That worked like a beaut, thanks. Should have thought about expr. Indeed. – thebluephantom Mar 08 '19 at 17:12

0 Answers0