I have csv file which contain userId, MovieId,Rating.I want to convert this file to containing label,features .
like in How to prepare data into a LibSVM format from DataFrame?
I need to separete rating column as afile and determine LabeledPoint for label.For applying random forest algorithm I need label column in file but it doesn't exit.
val pca = new PCA()
.setInputCol("features")
.setOutputCol("pcaFeatures")
.setK(3)
.fit(assembled_df)
val pcaTrainingData = pca.transform(assembled_df).select("id","features","pcaFeatures")
val labeled = pca.transform(assembled_df).rdd.map(row => LabeledPoint(
row.getAs[Double]("label"),
row.getAs[org.apache.spark.mllib.linalg.Vector]("pcaFeatures")
))
val numClasses = 10
val categoricalFeaturesInfo = Map[Int, Int]()
val numTrees = 10 // Use more in practice.
val featureSubsetStrategy = "auto" // Let the algorithm choose.
val impurity = "gini"
val maxDepth = 20
val maxBins = 32
val model = RandomForest.trainClassifier(labeled, numClasses, categoricalFeaturesInfo,
numTrees, featureSubsetStrategy, impurity, maxDepth, maxBins)
How to make label column?