1

I'm new to Scala and trying to create UDF function which would return a string of concatenated tuples identified by my UDF function.

It should look something like this, but have a couple of problems here, it does not like variable "fine" being defined outside of the block and doesn't want to change amount which is passed as a parameter.

val calculateFines: UserDefinedFunction = udf((ids: Array[Long], values: Array[Double], amount: Double, complain_id: Long) => {
       var fines
       ids.indices foreach { i => {
         val (id, value) = (ids(i), values(i))
         val penalty = if (value > amount)  amount else  value
         amount =  amount - penalty
         fines = fines + (amount, id, complain_id, penalty).toString()
         if (amount <= 0) 
           break
        }
       }
       return fines
     })
Krzysztof Atłasik
  • 21,985
  • 6
  • 54
  • 76
Lwica Gorska
  • 89
  • 1
  • 8
  • You can't create local variables without initializing them, and you can't reassign to a parameter. Also, there is no "break" keyword in Scala. An idiomatic way to write this in Scala would likely involve recursion. – Brian McCutchon Oct 15 '19 at 04:15

1 Answers1

1

You could make your code work with few fixes:

import scala.util.control.Breaks._ //we need this import to allow breaks since Scala doesn't support them out-of-box

val dysfunctional = udf((ids: Array[Long], values: Array[Double], amount: Double, complain_id: Long) => {
    var fines: String = "" //you need to initalize var
    var amountSum = amount //assign amount to var to allow to reassigment

    breakable {
      ids.indices foreach { i =>
        {
          val (id, value) = (ids(i), values(i))
          val penalty = if (value > amount) amount else value
          amountSum = amountSum - penalty
          fines = fines + (amount, id, complain_id, penalty)
          if (amount <= 0)
            break
        }
      }
    }
    fines
  })

This would work, but many people will frown upon it since it's very non-functional approach and Scala encourages writing functional code. You might try to change it to something like this:


val moreFunctional = (ids: Array[Long], values: Array[Double], amount: Double, complain_id: Long) => {

    val (_, fines) = (ids, values)
      .zipped // zip values and ids to single array of tuples
      .toStream //change it to stream to allow lazy computation
      .scanLeft((amount, "")) { //we pass tuple of amount and empty string as our initial state to scanLeft
        case ((amount, fines), (id, value)) => //second argument of scanLeft is function which receives previous state and currently processed element of array
          val penalty = if (value > amount) amount else value
          (amount, fines + (amount, id, complain_id, penalty).toString()) //here we passs next state for next iteration of scanLeft
      }
      .takeWhile { //we proceed with computations as long and amount is above zero
        case (amount, _) => amount > 0
      }
      .last //we're only interested in last state produced by scan left

    fines
  }
Krzysztof Atłasik
  • 21,985
  • 6
  • 54
  • 76