How to use existing trained model using LinearRegressionModel to work with SparkStreaming and predict data with it

Question

I trained data using LinearRegression and saved the model, now I am trying to use this in SparkStreaming and predict the data using it, but my program does not predict the data, other lines fetching data from stream and everything works in loop but prediction does not work, Can anyone help me with the prediction of data?

val conf = new SparkConf().setMaster("local[2]").setAppName("streamingApplication")    
val sc = new SparkContext(conf)
val ssc = new StreamingContext(sc,Seconds(60)) 



val finalData = data.map{
                      x => LabeledPoint(x.price , Vectors.dense(x.timestamp,x.volume,x.volume,x.price))
                    }                                   


        val model = LinearRegressionModel.load(sc, "linearReg")
        finalData.map{
            points => val prediction = model.predict(points.features)
            println(points.label)
            println(prediction)
            (points.label, prediction)
  }

println(points.label) println(prediction) Does not print anything — Chirag, May 03 '16 at 06:21
What's `data`? Is it possible there's nothing in it? Also you may be experiencing this: http://stackoverflow.com/questions/33225994/spark-losing-println-on-stdout — The Archetypal Paul, May 03 '16 at 06:30
no there is datat in it, i printed it val finalData = data.map{ x => LabeledPoint(x.price , Vectors.dense(x.timestamp,x.volume,x.volume,x.price)) } finalData.print() I used this and it prints data all the time but when it is passed for prediction ,nothing is printed — Chirag, May 03 '16 at 06:32
Try `finalData.map{println("Hello world")}` and see the question I linked to — The Archetypal Paul, May 03 '16 at 06:33
What worked? Please add some more detail for future readers of this question — The Archetypal Paul, May 03 '16 at 08:36
You mean the code was working, it was just the println output wasn't appearing? — The Archetypal Paul, May 03 '16 at 10:45
Yes it was, println was done separately and not where prediction is done — Chirag, May 03 '16 at 13:13

How to use existing trained model using LinearRegressionModel to work with SparkStreaming and predict data with it

0 Answers0