1

Input dataframe

    val ds = Seq((1,"play Framwork"),
  (2,"Spark framework"),
  (3,"spring framework")).toDF("id","subject")

Suppose my prefix value is prefix and suffix is suffix . Then i expect the following DataFrame

Expected dataframe

 val ds = Seq((1,"play Framwork",prefixplay Frameworksuffix),
  (2,"Spark framework",prefixSpark frameworksuffix),
  (3,"spring framework"),prefixspring frameworksuffix).toDF("id","subject",prefixsubjectsuffix)

So as you can see , i am expecting a new colunm to be created with name as prefix+columnName+suffix as prefixsubjectsuffix . Also the values of that column should be converted as my expected .

I am trying to use some udf to achieve this like as below

    import org.apache.spark.sql.functions.udf
  val concatPrefixSuffixWithColumnValues=
udf((column:String,prefix:String,suffix:String)=>
prefix.concat(column).concat(suffix))

But when i use it through the following statement gives some compilation error

dataset.withColumn(s"$prefixsubject$suffix",concatPrefixSuffixWithColumnValues("subject",prefix,suffix))

So how can i achieve this . Any references ??

adarsh kadameri
  • 89
  • 1
  • 3
  • 11

0 Answers0