Input dataframe
val ds = Seq((1,"play Framwork"),
(2,"Spark framework"),
(3,"spring framework")).toDF("id","subject")
Suppose my prefix
value is prefix and suffix
is suffix . Then i expect the following DataFrame
Expected dataframe
val ds = Seq((1,"play Framwork",prefixplay Frameworksuffix),
(2,"Spark framework",prefixSpark frameworksuffix),
(3,"spring framework"),prefixspring frameworksuffix).toDF("id","subject",prefixsubjectsuffix)
So as you can see , i am expecting a new colunm to be created with name as prefix+columnName+suffix
as prefixsubjectsuffix . Also the values
of that column should be converted as my expected .
I am trying to use some udf
to achieve this like as below
import org.apache.spark.sql.functions.udf
val concatPrefixSuffixWithColumnValues=
udf((column:String,prefix:String,suffix:String)=>
prefix.concat(column).concat(suffix))
But when i use it through the following statement gives some compilation error
dataset.withColumn(s"$prefixsubject$suffix",concatPrefixSuffixWithColumnValues("subject",prefix,suffix))
So how can i achieve this . Any references ??