1

By using group by, we get a GroupedData, how could I realize the normalization for each group of data seperatelly? Or for example, now I do something like

val df_list = trans.map(s => {
             println(s._1.toString)
             val scalerModel = scaler.fit(s._2)
             val scaledData = scalerModel.transform(s._2)
             scaledData})

where trans is an array of (string, df) and df is dataframe with "features"; I could realize in this way but not very efficient. Is there any better idea?

WU Zijun
  • 11
  • 2

0 Answers0